-- Logs begin at Fri 2022-08-26 11:33:48 UTC, end at Fri 2022-08-26 12:32:14 UTC. -- Aug 26 11:59:55.533577 dv-cinder-846303-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 26 11:59:55.542517 dv-cinder-846303-2-iscsi devstack@keystone.service[81701]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 26 11:59:55.547362 dv-cinder-846303-2-iscsi devstack@keystone.service[81701]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Aug 26 11:59:55 2022] *** Aug 26 11:59:55.547362 dv-cinder-846303-2-iscsi devstack@keystone.service[81701]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 26 11:59:55.547362 dv-cinder-846303-2-iscsi devstack@keystone.service[81701]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 26 11:59:55.547362 dv-cinder-846303-2-iscsi devstack@keystone.service[81701]: nodename: dv-cinder-846303-2-iscsi Aug 26 11:59:55.547362 dv-cinder-846303-2-iscsi devstack@keystone.service[81701]: machine: x86_64 Aug 26 11:59:55.547362 dv-cinder-846303-2-iscsi devstack@keystone.service[81701]: clock source: unix Aug 26 11:59:55.547825 dv-cinder-846303-2-iscsi devstack@keystone.service[81701]: pcre jit disabled Aug 26 11:59:55.547825 dv-cinder-846303-2-iscsi devstack@keystone.service[81701]: detected number of CPU cores: 4 Aug 26 11:59:55.547825 dv-cinder-846303-2-iscsi devstack@keystone.service[81701]: current working directory: / Aug 26 11:59:55.547825 dv-cinder-846303-2-iscsi devstack@keystone.service[81701]: detected binary path: /usr/bin/uwsgi-core Aug 26 11:59:55.547825 dv-cinder-846303-2-iscsi devstack@keystone.service[81701]: your processes number limit is 47680 Aug 26 11:59:55.547825 dv-cinder-846303-2-iscsi devstack@keystone.service[81701]: your memory page size is 4096 bytes Aug 26 11:59:55.547825 dv-cinder-846303-2-iscsi devstack@keystone.service[81701]: detected max file descriptor number: 2048 Aug 26 11:59:55.547825 dv-cinder-846303-2-iscsi devstack@keystone.service[81701]: lock engine: pthread robust mutexes Aug 26 11:59:55.547825 dv-cinder-846303-2-iscsi devstack@keystone.service[81701]: thunder lock: enabled Aug 26 11:59:55.548366 dv-cinder-846303-2-iscsi devstack@keystone.service[81701]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 26 11:59:55.548366 dv-cinder-846303-2-iscsi devstack@keystone.service[81701]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 26 11:59:55.591461 dv-cinder-846303-2-iscsi devstack@keystone.service[81701]: Python main interpreter initialized at 0x55a8c4c129c0 Aug 26 11:59:55.591461 dv-cinder-846303-2-iscsi devstack@keystone.service[81701]: python threads support enabled Aug 26 11:59:55.591461 dv-cinder-846303-2-iscsi devstack@keystone.service[81701]: your server socket listen backlog is limited to 100 connections Aug 26 11:59:55.591461 dv-cinder-846303-2-iscsi devstack@keystone.service[81701]: your mercy for graceful operations on workers is 90 seconds Aug 26 11:59:55.591845 dv-cinder-846303-2-iscsi devstack@keystone.service[81701]: mapped 403077 bytes (393 KB) for 2 cores Aug 26 11:59:55.591927 dv-cinder-846303-2-iscsi devstack@keystone.service[81701]: *** Operational MODE: preforking *** Aug 26 11:59:55.592200 dv-cinder-846303-2-iscsi devstack@keystone.service[81701]: *** uWSGI is running in multiple interpreter mode *** Aug 26 11:59:55.592200 dv-cinder-846303-2-iscsi devstack@keystone.service[81701]: spawned uWSGI master process (pid: 81701) Aug 26 11:59:55.592414 dv-cinder-846303-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 26 11:59:55.595871 dv-cinder-846303-2-iscsi devstack@keystone.service[81701]: spawned uWSGI worker 1 (pid: 81712, cores: 1) Aug 26 11:59:55.595871 dv-cinder-846303-2-iscsi devstack@keystone.service[81701]: spawned uWSGI worker 2 (pid: 81713, cores: 1) Aug 26 11:59:55.595871 dv-cinder-846303-2-iscsi devstack@keystone.service[81701]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 26 11:59:59.912325 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 26 11:59:59.912325 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: from cryptography.utils import int_from_bytes Aug 26 11:59:59.913121 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 26 11:59:59.913121 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: from cryptography.utils import int_from_bytes Aug 26 11:59:59.913266 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 26 11:59:59.913266 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: from cryptography.utils import int_from_bytes Aug 26 11:59:59.913705 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 26 11:59:59.913705 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: from cryptography.utils import int_from_bytes Aug 26 11:59:59.963296 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 26 11:59:59.963471 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 26 11:59:59.963665 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] command line args: [] {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 26 11:59:59.963758 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 26 11:59:59.963879 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 26 11:59:59.963965 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 26 11:59:59.964075 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] command line args: [] {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 26 11:59:59.964162 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] ================================================================================ {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 26 11:59:59.964329 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] admin_token = **** {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.964413 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 26 11:59:59.964511 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] ================================================================================ {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 26 11:59:59.964598 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] config_dir = [] {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.964745 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.964830 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] admin_token = **** {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.964926 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] config_source = [] {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.965003 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] config_dir = [] {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.965104 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] debug = True {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.965195 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.965302 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.966644 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] config_source = [] {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.966644 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] debug = True {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.966861 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.966945 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.967047 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.967119 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.967184 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.967291 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.967371 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.967448 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.967507 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] list_limit = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.967562 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.967624 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] log_config_append = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.967732 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] list_limit = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.967821 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.967898 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] log_config_append = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.967999 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] log_dir = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.968122 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.968197 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] log_file = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.968312 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] log_dir = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.968400 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.968471 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] log_file = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.968471 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.968644 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.968763 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.968838 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.968963 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.969038 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.969156 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.969231 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.969347 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.969430 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.969507 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.969624 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.969698 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.969819 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.969906 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.969985 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.970098 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.970196 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.970297 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.970403 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.970504 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.970623 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.970739 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.970840 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.970950 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.971057 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.971165 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.971308 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.971422 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.971525 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.61/identity {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.971625 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.61/identity {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.971736 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] publish_errors = False {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.971836 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] publish_errors = False {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.971941 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.972042 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.972148 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.972249 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.972352 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.972452 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.972529 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.972617 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.972699 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.972775 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.972897 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] standard_threads = False {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.972972 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] standard_threads = False {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.973090 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.973199 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.973311 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.973386 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.973462 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.973462 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] use_journal = False {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.973812 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.973892 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] use_json = False {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.973968 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] use_journal = False {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.974044 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] use_stderr = False {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.974119 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] use_json = False {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.974189 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] use_syslog = False {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.974264 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] use_stderr = False {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.974264 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] use_syslog = False {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.974391 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.974459 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 26 11:59:59.974533 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.974609 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.974689 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.974765 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.974840 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.974965 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.975046 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.975194 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.975305 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.975466 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.975545 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.975708 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.975786 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.975939 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.976028 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.976181 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.976259 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.976420 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.976497 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.976665 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.976753 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.976895 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.976973 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.977132 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.977209 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.977363 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.977449 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.977601 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.977679 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.977841 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.977918 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.978075 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.978165 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.978307 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.978385 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.978561 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.978646 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.978800 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.978952 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.979026 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.979095 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.979292 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.979379 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.979546 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.979627 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.979793 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.979870 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.980038 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.980116 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.980268 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.980353 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.980495 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.980573 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.980723 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.980807 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.980954 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.981033 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.981189 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.981273 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.981429 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.981506 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.981657 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.981736 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.981885 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.981963 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.982113 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.982196 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.982358 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.982435 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.982589 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.982671 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.982823 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.982906 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.983598 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.983686 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.983836 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.983916 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.984066 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.984143 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.984294 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.984374 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.984522 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.984606 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.984749 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.984834 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.984979 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.985057 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.985196 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.985274 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.985435 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.985512 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.985660 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.985738 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.985893 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.985972 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.986120 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.986198 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.986349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.986427 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.986579 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.986701 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.986815 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.986892 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.987040 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.987144 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.987296 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.987412 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.987515 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.987609 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.987750 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.987861 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.987983 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.988143 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.988229 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.988309 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.988427 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.988548 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.988659 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.988737 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.988890 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.988976 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.989124 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.989202 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.989353 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.989444 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.989582 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.989668 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.989826 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.989916 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.990054 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.990130 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.990286 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.990365 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.990520 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.990606 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.990754 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.990831 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.991007 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.991088 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.991272 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.991366 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.991514 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.991592 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.991749 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.991838 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.991984 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.992070 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.992216 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.992295 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.992451 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.992528 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.992682 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.992769 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.992915 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.992994 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.993153 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.993237 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.993432 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.993528 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.993629 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.993714 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.993860 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.993938 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.994093 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.994180 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.994325 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.994405 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.994558 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.994636 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.994790 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.994877 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.995024 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.995102 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.995292 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.995376 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.995539 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.995628 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.995773 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.995855 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.996006 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.996124 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.996244 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.996399 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.996476 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.996545 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.996714 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.996802 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.996927 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.997016 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.997154 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.997235 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.997382 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.997461 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.997609 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.997724 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.997841 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.997921 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.998066 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.998143 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.998295 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.998379 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.998526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.998641 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.999382 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.999493 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.999575 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.999813 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 11:59:59.999905 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.000069 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.000147 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.000328 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.000407 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.000568 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.000676 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.000797 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] auth.external = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.000886 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] auth.external = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.001027 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.001106 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.001317 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.001397 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.001550 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.001640 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.001778 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] auth.password = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.001857 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] auth.password = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.002006 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] auth.token = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.002085 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] auth.token = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.002239 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.002325 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.002474 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.002553 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.002717 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.002796 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.002945 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.003032 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.003188 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.003358 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.003460 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.003539 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.003689 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.003782 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.003924 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] credential.caching = True {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.004004 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] credential.caching = True {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.004168 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.004246 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.004415 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.004507 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.004656 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.004735 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.004895 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.004976 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.005135 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.005226 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.005366 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.005445 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.005609 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.005688 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.005859 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.005946 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.006094 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.006173 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.006342 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.006420 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.006619 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.006708 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.006861 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.006943 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.007120 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.007217 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.007382 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.007475 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.007635 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.007715 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.007869 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] federation.caching = True {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.008213 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] federation.caching = True {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.008371 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.008459 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.008614 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.008694 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.008859 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.008938 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.009084 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.009171 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.009326 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.009405 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.009558 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.009636 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.009806 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.009886 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.010037 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.010116 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.010287 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.010367 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.010521 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.010600 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.010761 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.010840 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.010994 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] identity.caching = True {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.011081 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] identity.caching = True {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.011255 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.011338 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.011494 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.011575 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.011729 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.011809 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.011963 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.012049 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.012206 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.012284 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.012434 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.012514 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.012667 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.012747 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.012909 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.012990 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.013156 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.013235 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.013382 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.013461 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.013611 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.013690 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.013836 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.013914 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.014067 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.014148 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.014334 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.014414 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.014583 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.014707 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.014878 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.014965 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.015171 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.015311 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.015466 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.015546 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.015701 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.015782 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.015935 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.016012 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.016163 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.016240 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.016397 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.016479 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.016623 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.016701 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.016854 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.016932 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.017091 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.017172 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.017325 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.017403 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.017570 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.017650 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.017797 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.017878 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.018041 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.018121 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.018289 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.018407 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.018523 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.018601 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.018762 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.018841 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.019005 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.019083 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.019250 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.019333 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.019488 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.019569 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.019835 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.019926 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.020073 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.020152 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.020309 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.020389 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.020550 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.020637 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.020783 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.020868 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.021018 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.021099 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.021264 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.021352 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.021510 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.021591 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.021740 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.021820 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.021968 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.022056 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.022213 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.022292 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.022457 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.022537 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.022693 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.022785 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.022928 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.023009 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.023165 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.023268 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.023418 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] ldap.user = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.023512 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] ldap.user = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.023661 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.023741 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.023914 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.023993 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.024145 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.024233 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.024391 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.024470 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.024633 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.024711 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.024878 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.024967 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.025112 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.025192 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.025340 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.025419 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.025572 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.025658 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.025805 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.025886 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.026039 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.026119 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.026268 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.026358 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.026508 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.026587 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.026752 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.026831 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.026992 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.027078 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.027252 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.027346 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.027496 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.027575 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.027725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.027813 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.027967 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.028049 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.028201 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.028281 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.028432 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.028520 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.028664 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.028744 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.028894 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.028974 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.029136 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.029223 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.029378 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.029456 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.029612 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.029690 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.029860 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.030445 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.030630 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.030813 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.030893 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.031076 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.031169 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.031322 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.031399 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.031556 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.031637 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.031794 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.031875 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.032041 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.032122 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.032269 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.032347 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.032497 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.032575 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.032730 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.032812 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] resource.caching = True {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.032968 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] resource.caching = True {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.033047 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.033212 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.033290 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.033450 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.033531 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.033679 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.033758 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.033924 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.034004 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.034158 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.034239 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.034395 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.034473 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.034629 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.034707 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.034898 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.034978 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.035080 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.035160 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] role.caching = True {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.035334 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] role.caching = True {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.035433 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] role.driver = sql {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.035571 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.035650 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] role.driver = sql {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.035795 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.035873 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.036040 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.036131 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.036284 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.036365 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.036528 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.036607 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.036767 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.036854 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.037009 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.037093 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.037252 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.037331 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.037490 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.037576 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.037722 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.037801 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.037946 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.038024 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.038186 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.038274 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.038429 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.038509 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.038671 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.038750 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.038913 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.039059 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.039320 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.039404 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.039553 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.039632 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.039798 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.039891 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.040042 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.040122 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.040285 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.040365 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.040525 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.040613 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.040752 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.040831 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.041001 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.041080 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.041243 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.041323 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.041475 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.041555 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.041704 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.041784 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.041933 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.042014 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.042159 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.042238 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.042405 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.042485 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.042657 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.042737 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.042896 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.042976 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.043129 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.043256 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.043382 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.043469 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.043608 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.043687 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] token.caching = True {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.043841 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] token.caching = True {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.043921 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.044081 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.044162 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.044317 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.044395 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.044548 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.044633 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.044803 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.044884 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.045037 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.045115 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.045268 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.045346 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.045515 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.046128 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.046318 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.046555 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.046636 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.046786 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.046874 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.047019 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.047099 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.047272 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.047357 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.047518 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.047611 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.047796 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.047876 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.048023 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.048112 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.048260 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.048349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.048520 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.048599 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.048774 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] database.connection = **** {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.048856 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] database.connection = **** {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.049011 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.049091 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.049253 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.049333 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.049487 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.049568 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.049725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.049804 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.049958 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.050038 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.050193 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.050273 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.050422 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.050501 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.050654 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.050734 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.050895 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.050977 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.051127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.051220 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.051394 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.051472 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.051625 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.051705 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.051868 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.051948 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.052100 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.052180 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.052340 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.052421 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.052565 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.052645 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.052804 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.052883 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.053035 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.053116 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 26 12:00:00.053187 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81712) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 26 12:00:00.053257 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81713) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 26 12:00:00.082586 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 12:00:00.082816 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 12:00:00.082903 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 12:00:00.083042 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 12:00:00.083127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 12:00:00.083289 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 12:00:00.091462 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 12:00:00.091701 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 12:00:00.120779 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 12:00:00.121008 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 12:00:00.121167 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 12:00:00.121405 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 12:00:00.175699 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 12:00:00.175908 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 12:00:00.224772 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 12:00:00.225010 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 12:00:00.225372 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 12:00:00.225608 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 12:00:00.383513 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 12:00:00.383745 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 12:00:00.383971 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 12:00:00.384195 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 12:00:00.384301 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 12:00:00.384416 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 12:00:00.384541 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 12:00:00.384624 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 12:00:00.384765 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 12:00:00.384984 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 12:00:00.385200 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 12:00:00.385417 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 12:00:00.408826 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 12:00:00.408940 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 12:00:00.409104 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 12:00:00.409289 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 12:00:00.409423 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 12:00:00.409557 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 12:00:00.409703 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 12:00:00.409834 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 12:00:00.409973 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 12:00:00.410104 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 12:00:00.410235 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 12:00:00.410313 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 12:00:00.410386 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 12:00:00.410386 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 12:00:00.421372 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 12:00:00.421654 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 12:00:00.434040 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 12:00:00.434283 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 12:00:00.434506 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 12:00:00.434607 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 12:00:00.434730 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 12:00:00.434814 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 12:00:00.434950 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 12:00:00.435033 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 12:00:00.435169 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 12:00:00.435281 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 12:00:00.435419 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81712) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 12:00:00.435503 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 12:00:00.435655 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 12:00:00.435872 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81713) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 26 12:00:00.452741 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.452956 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 26 12:00:00.453157 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.453309 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.453391 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 26 12:00:00.453514 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 26 12:00:00.453697 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.453779 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.453900 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 26 12:00:00.454043 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.454175 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.454437 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.454556 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.454820 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.454941 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.455224 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.464291 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.464565 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.464648 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.464840 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.464922 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.465109 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.465192 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.465393 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.472791 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81712) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 12:00:00.473105 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81713) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 12:00:00.475736 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81712) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 12:00:00.476020 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81713) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 12:00:00.476105 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.476304 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.476386 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.476591 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.476686 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.476875 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.476958 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.477139 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.477222 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.477400 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.477481 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.477669 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.477751 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.477941 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.478075 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.478321 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.478405 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.478594 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.478676 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.478864 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.501539 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.501637 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.502992 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81713) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 12:00:00.503077 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81712) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 12:00:00.503298 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.503400 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.508255 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81713) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 12:00:00.508351 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81712) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 12:00:00.508542 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.508626 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.508820 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.508904 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.515795 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81713) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 12:00:00.515921 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81712) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 12:00:00.516076 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.516164 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.520191 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81713) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 12:00:00.520285 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81712) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 12:00:00.520489 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.520573 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.520768 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.520865 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.521053 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.521135 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.521328 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.521410 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.521602 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.521685 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.521870 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.521951 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.522135 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.522216 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.539811 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.540092 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.540176 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.540428 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.543984 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81713) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 12:00:00.544370 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81712) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 12:00:00.547947 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.548245 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.548367 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.548676 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.552491 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81713) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 12:00:00.552927 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81712) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 12:00:00.555672 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81713) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 12:00:00.556129 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81712) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 12:00:00.558906 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.559228 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.559393 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.559525 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.559698 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.559814 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.559985 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.560103 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.560269 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.560400 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.560560 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.560861 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.576853 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81713) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 12:00:00.577157 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.577356 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81712) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 12:00:00.577590 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.577673 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.577870 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.577953 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.578178 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.584791 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.585186 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.586331 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.586618 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.586729 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.587011 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.589289 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81713) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 12:00:00.589578 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.589713 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81712) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 12:00:00.589867 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.590005 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.590148 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.590299 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.590422 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.590585 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.590865 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.602068 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81713) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 12:00:00.602352 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.602448 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81712) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 12:00:00.602633 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.602716 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.602911 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.603004 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.603192 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.603299 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.603490 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.603574 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.603768 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.603864 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.604076 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.619736 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81713) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 12:00:00.620138 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81712) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 12:00:00.622758 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81713) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 12:00:00.623192 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81712) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 12:00:00.626349 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.626674 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.627812 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.628158 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.629289 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81713) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 12:00:00.629575 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.629661 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81712) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 12:00:00.629853 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.629937 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.630189 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.636787 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.637143 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.638341 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81713) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 12:00:00.638639 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81712) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 12:00:00.641387 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.641581 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.641668 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.641876 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.641960 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.642168 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.642251 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.642465 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.650181 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81713) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 12:00:00.650487 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.650569 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81712) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 12:00:00.650770 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.650866 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.651089 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.657802 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81713) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 12:00:00.658087 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.658171 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81712) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 26 12:00:00.658373 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.658456 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.658647 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.658729 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.658916 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.659001 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.659184 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.659284 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.659477 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.659560 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.659752 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.659846 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.660017 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.660108 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.660292 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.660384 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.660564 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.660645 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.660830 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.660912 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.661106 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.661190 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.661362 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81713) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.661444 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.661622 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81712) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 26 12:00:00.696485 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55a8c4c129c0 pid: 81713 (default app) Aug 26 12:00:00.697097 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55a8c4c129c0 pid: 81712 (default app) Aug 26 12:00:00.699889 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b099496-3660-4fac-bc02-30c13b3fa0d7 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:00.700174 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b099496-3660-4fac-bc02-30c13b3fa0d7 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:00.700444 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b099496-3660-4fac-bc02-30c13b3fa0d7 None None] PATH_INFO: `/v3` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:00.704624 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1/1] 10.222.0.61 () {50 vars in 791 bytes} [Fri Aug 26 12:00:00 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:07.313531 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7065c30-59c8-4609-b616-f3381e6e7d3f None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:07.314135 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7065c30-59c8-4609-b616-f3381e6e7d3f None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:07.314135 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7065c30-59c8-4609-b616-f3381e6e7d3f None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:07.318364 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1/2] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:00:07 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 12:00:07.323678 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b0e234-f6f7-4cf9-880f-219a642d48c3 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:07.324093 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b0e234-f6f7-4cf9-880f-219a642d48c3 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:07.324228 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b0e234-f6f7-4cf9-880f-219a642d48c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:07.394660 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: INFO dbcounter [None req-24b0e234-f6f7-4cf9-880f-219a642d48c3 None None] Registered counter for database keystone Aug 26 12:00:07.413518 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG oslo_db.sqlalchemy.engines [None req-24b0e234-f6f7-4cf9-880f-219a642d48c3 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81713) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 26 12:00:07.419982 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writer thread running {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 26 12:00:07.598079 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG passlib.handlers.bcrypt [None req-24b0e234-f6f7-4cf9-880f-219a642d48c3 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81713) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 26 12:00:07.598408 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG passlib.handlers.bcrypt [None req-24b0e234-f6f7-4cf9-880f-219a642d48c3 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81713) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 26 12:00:07.653294 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-24b0e234-f6f7-4cf9-880f-219a642d48c3 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:00:07.655483 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-24b0e234-f6f7-4cf9-880f-219a642d48c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:00:07.783143 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 2/3] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:00:07 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 460 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 26 12:00:08.271136 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7065c30-59c8-4609-b616-f3381e6e7d3f None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:08.339853 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: INFO dbcounter [None req-c7065c30-59c8-4609-b616-f3381e6e7d3f None None] Registered counter for database keystone Aug 26 12:00:08.361014 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG oslo_db.sqlalchemy.engines [None req-c7065c30-59c8-4609-b616-f3381e6e7d3f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81712) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 26 12:00:08.367004 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writer thread running {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 26 12:00:08.486430 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7065c30-59c8-4609-b616-f3381e6e7d3f None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:08.490659 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a834446a-4167-40e7-a7d3-a45219213166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:08.491299 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a834446a-4167-40e7-a7d3-a45219213166 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:08.491582 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a834446a-4167-40e7-a7d3-a45219213166 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:08.491853 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a834446a-4167-40e7-a7d3-a45219213166 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:08.500403 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a834446a-4167-40e7-a7d3-a45219213166 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:08.552411 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: WARNING keystone.common.rbac_enforcer.enforcer [None req-a834446a-4167-40e7-a7d3-a45219213166 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 26 12:00:08.552694 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a834446a-4167-40e7-a7d3-a45219213166 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:08.562417 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 2/4] 10.222.0.61 () {58 vars in 1324 bytes} [Fri Aug 26 12:00:08 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 26 12:00:08.568357 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b0e234-f6f7-4cf9-880f-219a642d48c3 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:08.583346 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b0e234-f6f7-4cf9-880f-219a642d48c3 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:08.587783 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3786803e-ebea-4b3c-b2df-9ac0936e8ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:08.588875 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3786803e-ebea-4b3c-b2df-9ac0936e8ce5 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:08.589161 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3786803e-ebea-4b3c-b2df-9ac0936e8ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:08.589435 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3786803e-ebea-4b3c-b2df-9ac0936e8ce5 admin admin] PATH_INFO: `/v3/projects/b98963fff98a48a1b4cae11131b453da` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:08.590979 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3786803e-ebea-4b3c-b2df-9ac0936e8ce5 admin admin] RBAC: Authorizing `identity:get_project(project_id=b98963fff98a48a1b4cae11131b453da)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:08.645146 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: WARNING keystone.common.rbac_enforcer.enforcer [None req-3786803e-ebea-4b3c-b2df-9ac0936e8ce5 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 26 12:00:08.645505 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3786803e-ebea-4b3c-b2df-9ac0936e8ce5 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:08.649931 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 3/5] 10.222.0.61 () {56 vars in 1207 bytes} [Fri Aug 26 12:00:08 2022] GET /identity/v3/projects/b98963fff98a48a1b4cae11131b453da => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:00:10.525016 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec9a836-632a-4eac-8615-1cf27fb74f71 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:10.525553 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec9a836-632a-4eac-8615-1cf27fb74f71 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:10.525553 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec9a836-632a-4eac-8615-1cf27fb74f71 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:10.529712 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 3/6] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:00:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:00:10.535381 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d29cfe-99d9-4a78-8333-1637141a9a72 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:10.535845 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d29cfe-99d9-4a78-8333-1637141a9a72 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:10.535928 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d29cfe-99d9-4a78-8333-1637141a9a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:10.555556 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-48d29cfe-99d9-4a78-8333-1637141a9a72 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:00:10.557050 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-48d29cfe-99d9-4a78-8333-1637141a9a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:00:10.565336 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 4/7] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:00:10 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 26 12:00:10.832309 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec9a836-632a-4eac-8615-1cf27fb74f71 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:10.845072 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec9a836-632a-4eac-8615-1cf27fb74f71 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:10.849230 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075484e7-3fc9-4ac1-800c-9e813c1c6406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:10.849818 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075484e7-3fc9-4ac1-800c-9e813c1c6406 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:10.850096 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075484e7-3fc9-4ac1-800c-9e813c1c6406 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:10.850368 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075484e7-3fc9-4ac1-800c-9e813c1c6406 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:10.857940 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075484e7-3fc9-4ac1-800c-9e813c1c6406 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:10.860307 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075484e7-3fc9-4ac1-800c-9e813c1c6406 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:10.868893 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 4/8] 10.222.0.61 () {58 vars in 1324 bytes} [Fri Aug 26 12:00:10 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 12:00:10.874242 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d29cfe-99d9-4a78-8333-1637141a9a72 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:10.883603 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d29cfe-99d9-4a78-8333-1637141a9a72 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:10.887901 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eefda9-fcf2-465c-9948-c8c347c2003c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:10.888517 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eefda9-fcf2-465c-9948-c8c347c2003c admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:10.888794 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eefda9-fcf2-465c-9948-c8c347c2003c admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:10.889071 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eefda9-fcf2-465c-9948-c8c347c2003c admin admin] PATH_INFO: `/v3/users/d101ea12e0cd40e0b25022bf649ac734` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:10.891162 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eefda9-fcf2-465c-9948-c8c347c2003c admin admin] RBAC: Authorizing `identity:get_user(user_id=d101ea12e0cd40e0b25022bf649ac734)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:10.893016 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eefda9-fcf2-465c-9948-c8c347c2003c admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:10.894531 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 5/9] 10.222.0.61 () {56 vars in 1198 bytes} [Fri Aug 26 12:00:10 2022] GET /identity/v3/users/d101ea12e0cd40e0b25022bf649ac734 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:12.849405 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef80924-7646-4ff9-8b24-ed57ee799ec6 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:12.849931 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef80924-7646-4ff9-8b24-ed57ee799ec6 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:12.849931 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef80924-7646-4ff9-8b24-ed57ee799ec6 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:12.851325 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 5/10] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:00:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:00:12.856498 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac906853-8acf-43e1-8faf-0a8df7948a62 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:12.856898 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac906853-8acf-43e1-8faf-0a8df7948a62 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:12.857051 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac906853-8acf-43e1-8faf-0a8df7948a62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:12.875285 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-ac906853-8acf-43e1-8faf-0a8df7948a62 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:00:12.876454 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-ac906853-8acf-43e1-8faf-0a8df7948a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:00:12.897326 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c002af0-452a-4f27-9563-db557fa67b7e None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:12.897616 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c002af0-452a-4f27-9563-db557fa67b7e None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:12.897897 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c002af0-452a-4f27-9563-db557fa67b7e None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:12.899224 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 6/11] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:00:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 12:00:12.904355 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b55cf8c-ec27-44d7-8b20-3e03d72808f0 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:12.905187 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b55cf8c-ec27-44d7-8b20-3e03d72808f0 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:12.905466 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b55cf8c-ec27-44d7-8b20-3e03d72808f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:12.947767 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG passlib.handlers.bcrypt [None req-8b55cf8c-ec27-44d7-8b20-3e03d72808f0 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81712) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 26 12:00:12.948188 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG passlib.handlers.bcrypt [None req-8b55cf8c-ec27-44d7-8b20-3e03d72808f0 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81712) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 26 12:00:12.956816 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 6/12] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:00:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 12:00:12.994129 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-8b55cf8c-ec27-44d7-8b20-3e03d72808f0 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:00:12.995802 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-8b55cf8c-ec27-44d7-8b20-3e03d72808f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:00:13.095890 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 7/13] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:00:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 12:00:13.224550 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac906853-8acf-43e1-8faf-0a8df7948a62 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:13.247513 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:13.326345 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac906853-8acf-43e1-8faf-0a8df7948a62 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:13.359698 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee8ee24-f435-4e9d-a003-9a9ce5d9a0a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:13.360330 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee8ee24-f435-4e9d-a003-9a9ce5d9a0a0 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:13.360607 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee8ee24-f435-4e9d-a003-9a9ce5d9a0a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:13.360876 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee8ee24-f435-4e9d-a003-9a9ce5d9a0a0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:13.363732 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b55cf8c-ec27-44d7-8b20-3e03d72808f0 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:13.372290 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee8ee24-f435-4e9d-a003-9a9ce5d9a0a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:00:13.380765 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-3ee8ee24-f435-4e9d-a003-9a9ce5d9a0a0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 12:00:13.380765 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:00:13.380765 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:00:13.380765 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:00:13.380765 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:00:13.380765 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:00:13.380765 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:00:13.380765 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:00:13.380765 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:00:13.380765 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:00:13.380765 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:00:13.380765 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:00:13.380765 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:00:13.380765 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:00:13.380765 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:00:13.380765 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:00:13.381872 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:00:13.381872 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:00:13.381872 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:00:13.381872 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:00:13.381872 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:00:13.381872 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:00:13.381872 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:00:13.381872 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:00:13.381872 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:00:13.381872 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:00:13.381872 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:00:13.381872 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:00:13.381872 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:00:13.381872 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:00:13.381872 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:00:13.381872 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:00:13.383411 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:00:13.383411 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:00:13.383411 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:00:13.383411 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:00:13.383411 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:00:13.383411 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:00:13.383411 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:00:13.383411 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:00:13.383411 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:00:13.383411 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:00:13.383411 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 12:00:13.383411 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:00:13.383411 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 7/14] 10.222.0.61 () {56 vars in 1096 bytes} [Fri Aug 26 12:00:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:00:13.387723 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee8ee24-f435-4e9d-a003-9a9ce5d9a0a0 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:13.468658 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:13.469609 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b55cf8c-ec27-44d7-8b20-3e03d72808f0 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:13.472575 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:13.486818 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee8ee24-f435-4e9d-a003-9a9ce5d9a0a0 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:13.508899 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bf145b-9b7f-47aa-9ef0-0602aeee90e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:13.509683 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf145b-9b7f-47aa-9ef0-0602aeee90e4 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:13.509971 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf145b-9b7f-47aa-9ef0-0602aeee90e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:13.510507 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf145b-9b7f-47aa-9ef0-0602aeee90e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:13.522021 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3ff903-fb04-418a-9ea2-f27c15313c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:13.522624 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3ff903-fb04-418a-9ea2-f27c15313c21 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:13.522904 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3ff903-fb04-418a-9ea2-f27c15313c21 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:13.523179 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3ff903-fb04-418a-9ea2-f27c15313c21 None admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:13.523875 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3ff903-fb04-418a-9ea2-f27c15313c21 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:00:13.524176 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3ff903-fb04-418a-9ea2-f27c15313c21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:13.525580 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3ff903-fb04-418a-9ea2-f27c15313c21 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:13.537776 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 8/15] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:00:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:00:13.543335 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3ff903-fb04-418a-9ea2-f27c15313c21 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:13.581536 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bf145b-9b7f-47aa-9ef0-0602aeee90e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:13.583059 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bf145b-9b7f-47aa-9ef0-0602aeee90e4 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:13.641603 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3ff903-fb04-418a-9ea2-f27c15313c21 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:13.675818 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e9ed50-16ed-4a68-8d6d-e2028cafb6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:13.676444 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e9ed50-16ed-4a68-8d6d-e2028cafb6c9 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:13.676732 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e9ed50-16ed-4a68-8d6d-e2028cafb6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:13.677010 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e9ed50-16ed-4a68-8d6d-e2028cafb6c9 None admin] PATH_INFO: `/v3/users/d101ea12e0cd40e0b25022bf649ac734` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:13.678140 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e9ed50-16ed-4a68-8d6d-e2028cafb6c9 None admin] RBAC: Authorizing `identity:get_user(user_id=d101ea12e0cd40e0b25022bf649ac734)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:13.679618 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e9ed50-16ed-4a68-8d6d-e2028cafb6c9 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:13.680085 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 8/16] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:00:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 12:00:13.685658 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 9/17] 10.222.0.61 () {56 vars in 1177 bytes} [Fri Aug 26 12:00:13 2022] GET /identity/v3/users/d101ea12e0cd40e0b25022bf649ac734 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:00:13.685791 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bf145b-9b7f-47aa-9ef0-0602aeee90e4 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:13.686615 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e9ed50-16ed-4a68-8d6d-e2028cafb6c9 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:13.689004 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:13.701387 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:13.787215 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e9ed50-16ed-4a68-8d6d-e2028cafb6c9 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:13.787519 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bf145b-9b7f-47aa-9ef0-0602aeee90e4 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:13.820549 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f8aa98-0c11-4851-8db4-de27d145b647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:13.821362 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82428eb-1602-45d4-b1ed-ce19e579ba8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:13.821468 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8aa98-0c11-4851-8db4-de27d145b647 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:13.821669 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8aa98-0c11-4851-8db4-de27d145b647 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:13.821948 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8aa98-0c11-4851-8db4-de27d145b647 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:13.822040 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82428eb-1602-45d4-b1ed-ce19e579ba8d None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:13.822256 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82428eb-1602-45d4-b1ed-ce19e579ba8d None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:13.822531 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82428eb-1602-45d4-b1ed-ce19e579ba8d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:13.823666 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82428eb-1602-45d4-b1ed-ce19e579ba8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:13.825117 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82428eb-1602-45d4-b1ed-ce19e579ba8d None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:13.826690 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 10/18] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:00:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:00:13.832414 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82428eb-1602-45d4-b1ed-ce19e579ba8d None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:13.840589 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f8aa98-0c11-4851-8db4-de27d145b647 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:13.842052 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f8aa98-0c11-4851-8db4-de27d145b647 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:13.861385 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-11f8aa98-0c11-4851-8db4-de27d145b647 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:00:13.861385 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:00:13.861385 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:00:13.861385 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:00:13.861385 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:00:13.861385 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:00:13.861385 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:00:13.861385 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:00:13.861385 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:00:13.861385 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:00:13.861385 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:00:13.861385 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:00:13.861385 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:00:13.861385 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:00:13.861385 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:00:13.861385 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:00:13.862728 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:00:13.862728 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:00:13.862728 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:00:13.862728 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:00:13.862728 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:00:13.862728 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:00:13.862728 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:00:13.862728 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:00:13.862728 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:00:13.862728 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:00:13.862728 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:00:13.862728 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:00:13.862728 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:00:13.862728 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:00:13.862728 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:00:13.862728 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:00:13.862728 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:00:13.863615 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:00:13.863615 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:00:13.863615 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:00:13.863615 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 9/19] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:00:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:00:13.868224 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f8aa98-0c11-4851-8db4-de27d145b647 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:13.912504 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:13.924980 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:13.932282 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82428eb-1602-45d4-b1ed-ce19e579ba8d None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:13.965823 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff71d16b-f303-426d-a7fc-d1cc30ad617c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:13.966426 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff71d16b-f303-426d-a7fc-d1cc30ad617c None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:13.966745 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff71d16b-f303-426d-a7fc-d1cc30ad617c None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:13.967028 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff71d16b-f303-426d-a7fc-d1cc30ad617c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:13.968426 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff71d16b-f303-426d-a7fc-d1cc30ad617c None admin] RBAC: Adding query filter params (role.id=70ec1ba484714c8ca3adb8fe5845107c, scope.domain.id=default, user.id=d101ea12e0cd40e0b25022bf649ac734) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:00:13.968735 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff71d16b-f303-426d-a7fc-d1cc30ad617c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:13.969820 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f8aa98-0c11-4851-8db4-de27d145b647 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:13.970185 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff71d16b-f303-426d-a7fc-d1cc30ad617c None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:13.977099 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 11/20] 10.222.0.61 () {56 vars in 1322 bytes} [Fri Aug 26 12:00:13 2022] GET /identity/v3/role_assignments?user.id=d101ea12e0cd40e0b25022bf649ac734&scope.domain.id=default&role.id=70ec1ba484714c8ca3adb8fe5845107c => generated 233 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:00:14.005069 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bea47a4-162e-43b8-8c80-ca94b0155fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:14.005691 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bea47a4-162e-43b8-8c80-ca94b0155fe2 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:14.005974 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bea47a4-162e-43b8-8c80-ca94b0155fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:14.006245 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bea47a4-162e-43b8-8c80-ca94b0155fe2 None admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:14.006903 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bea47a4-162e-43b8-8c80-ca94b0155fe2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:00:14.007229 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bea47a4-162e-43b8-8c80-ca94b0155fe2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:14.008910 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bea47a4-162e-43b8-8c80-ca94b0155fe2 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:14.031793 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 10/21] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:00:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:00:16.014760 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca815456-16a2-4f1e-a714-3205b0e401c3 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:16.015372 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca815456-16a2-4f1e-a714-3205b0e401c3 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:16.015372 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca815456-16a2-4f1e-a714-3205b0e401c3 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:16.016579 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 12/22] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:00:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:00:16.033139 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb28cfd-5c93-4044-b568-d22f64ada35a None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:16.033611 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb28cfd-5c93-4044-b568-d22f64ada35a None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:16.033682 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb28cfd-5c93-4044-b568-d22f64ada35a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:16.064194 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-fbb28cfd-5c93-4044-b568-d22f64ada35a None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:00:16.065414 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-fbb28cfd-5c93-4044-b568-d22f64ada35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:00:16.157520 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 11/23] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:00:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:00:16.538664 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23edef3-1490-483e-b54b-c5c75bbd75d5 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:16.539023 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23edef3-1490-483e-b54b-c5c75bbd75d5 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:16.539260 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23edef3-1490-483e-b54b-c5c75bbd75d5 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:16.540585 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 13/24] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:00:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:00:16.546519 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a67e4-6952-4c0d-b0bd-daba83eb6c5a None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:16.546810 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a67e4-6952-4c0d-b0bd-daba83eb6c5a None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:16.547089 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14a67e4-6952-4c0d-b0bd-daba83eb6c5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:16.577416 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-e14a67e4-6952-4c0d-b0bd-daba83eb6c5a None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:00:16.578604 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-e14a67e4-6952-4c0d-b0bd-daba83eb6c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:00:16.620440 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23edef3-1490-483e-b54b-c5c75bbd75d5 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:16.673300 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:16.704997 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 12/25] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:00:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 12:00:16.706280 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a0013-5361-4514-a67b-c696f697dc0d None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:16.706570 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a0013-5361-4514-a67b-c696f697dc0d None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:16.713264 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a0013-5361-4514-a67b-c696f697dc0d None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:16.715177 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 13/26] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:00:16 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:00:16.716566 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153776a2-c4f0-4cd8-9449-d1898d054e9e None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:16.716862 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153776a2-c4f0-4cd8-9449-d1898d054e9e None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:16.717866 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153776a2-c4f0-4cd8-9449-d1898d054e9e None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:16.719678 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 14/27] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:00:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:00:16.721000 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05907a8-f627-4ebf-a5e4-46cf332f9832 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:16.727754 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05907a8-f627-4ebf-a5e4-46cf332f9832 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:16.728093 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05907a8-f627-4ebf-a5e4-46cf332f9832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:16.763613 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-a05907a8-f627-4ebf-a5e4-46cf332f9832 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:00:16.765385 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-a05907a8-f627-4ebf-a5e4-46cf332f9832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:00:16.774382 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23edef3-1490-483e-b54b-c5c75bbd75d5 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:16.822994 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7d857b-8403-4bdf-bcfc-9e3802a6ee6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:16.824016 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d857b-8403-4bdf-bcfc-9e3802a6ee6f None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:16.824466 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d857b-8403-4bdf-bcfc-9e3802a6ee6f None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:16.824885 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d857b-8403-4bdf-bcfc-9e3802a6ee6f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:16.836508 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7d857b-8403-4bdf-bcfc-9e3802a6ee6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:00:16.841979 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-5b7d857b-8403-4bdf-bcfc-9e3802a6ee6f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 12:00:16.841979 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:00:16.841979 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:00:16.841979 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:00:16.841979 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:00:16.841979 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:00:16.841979 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:00:16.841979 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:00:16.841979 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:00:16.841979 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:00:16.841979 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:00:16.841979 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:00:16.841979 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:00:16.841979 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:00:16.841979 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:00:16.841979 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:00:16.843537 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:00:16.843537 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:00:16.843537 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:00:16.843537 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:00:16.843537 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:00:16.843537 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:00:16.843537 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:00:16.843537 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:00:16.843537 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:00:16.843537 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:00:16.843537 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:00:16.843537 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:00:16.843537 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:00:16.843537 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:00:16.843537 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:00:16.843537 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:00:16.844486 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:00:16.844486 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:00:16.844486 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:00:16.844486 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:00:16.844486 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:00:16.844486 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:00:16.844486 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:00:16.844486 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:00:16.844486 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:00:16.844486 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:00:16.844486 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 12:00:16.844486 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:00:16.846355 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 14/28] 10.222.0.61 () {56 vars in 1096 bytes} [Fri Aug 26 12:00:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 12:00:16.847651 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab786562-d1a6-4a16-8710-1161b1839d2c None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:16.848073 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab786562-d1a6-4a16-8710-1161b1839d2c None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:16.848585 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab786562-d1a6-4a16-8710-1161b1839d2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:16.856060 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:16.869319 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-ab786562-d1a6-4a16-8710-1161b1839d2c None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:00:16.870756 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-ab786562-d1a6-4a16-8710-1161b1839d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:00:16.893316 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 15/29] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:00:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:00:16.894177 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05907a8-f627-4ebf-a5e4-46cf332f9832 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:16.975229 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 15/30] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:00:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:00:17.088981 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05907a8-f627-4ebf-a5e4-46cf332f9832 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:17.101570 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:17.143900 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878387c1-8460-484f-8ddf-2633c91ab42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:17.144508 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878387c1-8460-484f-8ddf-2633c91ab42c None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:17.144783 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878387c1-8460-484f-8ddf-2633c91ab42c None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:17.145047 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878387c1-8460-484f-8ddf-2633c91ab42c None admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:17.145712 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878387c1-8460-484f-8ddf-2633c91ab42c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:00:17.146012 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878387c1-8460-484f-8ddf-2633c91ab42c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:17.147411 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878387c1-8460-484f-8ddf-2633c91ab42c None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:17.166913 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 16/31] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:00:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:17.173304 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab786562-d1a6-4a16-8710-1161b1839d2c None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:17.216969 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878387c1-8460-484f-8ddf-2633c91ab42c None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:17.293227 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab786562-d1a6-4a16-8710-1161b1839d2c None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:17.330801 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dda205-b312-461b-97bf-79875f8dc017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:17.331704 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dda205-b312-461b-97bf-79875f8dc017 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:17.332156 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dda205-b312-461b-97bf-79875f8dc017 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:17.332613 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dda205-b312-461b-97bf-79875f8dc017 None admin] PATH_INFO: `/v3/users/d101ea12e0cd40e0b25022bf649ac734` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:17.333910 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dda205-b312-461b-97bf-79875f8dc017 None admin] RBAC: Authorizing `identity:get_user(user_id=d101ea12e0cd40e0b25022bf649ac734)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:17.335526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dda205-b312-461b-97bf-79875f8dc017 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:17.337251 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 16/32] 10.222.0.61 () {56 vars in 1177 bytes} [Fri Aug 26 12:00:17 2022] GET /identity/v3/users/d101ea12e0cd40e0b25022bf649ac734 => generated 247 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:17.344615 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dda205-b312-461b-97bf-79875f8dc017 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:17.347563 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:17.457539 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dda205-b312-461b-97bf-79875f8dc017 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:17.465464 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878387c1-8460-484f-8ddf-2633c91ab42c None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:17.501700 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b58d92-3a54-4c93-87e6-ba5ac6b37a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:17.502389 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b58d92-3a54-4c93-87e6-ba5ac6b37a28 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:17.502669 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b58d92-3a54-4c93-87e6-ba5ac6b37a28 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:17.502947 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b58d92-3a54-4c93-87e6-ba5ac6b37a28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:17.503432 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af0e8b3-c287-4ba3-9063-f9fa41bec895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:17.504440 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af0e8b3-c287-4ba3-9063-f9fa41bec895 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:17.504878 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af0e8b3-c287-4ba3-9063-f9fa41bec895 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:17.505449 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af0e8b3-c287-4ba3-9063-f9fa41bec895 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:17.507093 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af0e8b3-c287-4ba3-9063-f9fa41bec895 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:17.508967 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af0e8b3-c287-4ba3-9063-f9fa41bec895 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:17.511159 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 17/33] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:00:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:00:17.512537 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af0e8b3-c287-4ba3-9063-f9fa41bec895 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:17.514914 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b58d92-3a54-4c93-87e6-ba5ac6b37a28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:17.516472 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b58d92-3a54-4c93-87e6-ba5ac6b37a28 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:17.532521 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-f3b58d92-3a54-4c93-87e6-ba5ac6b37a28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:00:17.532521 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:00:17.532521 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:00:17.532521 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:00:17.532521 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:00:17.532521 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:00:17.532521 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:00:17.532521 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:00:17.532521 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:00:17.532521 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:00:17.532521 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:00:17.532521 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:00:17.532521 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:00:17.532521 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:00:17.532521 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:00:17.532521 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:00:17.533630 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:00:17.533630 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:00:17.533630 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:00:17.533630 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:00:17.533630 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:00:17.533630 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:00:17.533630 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:00:17.533630 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:00:17.533630 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:00:17.533630 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:00:17.533630 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:00:17.533630 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:00:17.533630 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:00:17.533630 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:00:17.533630 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:00:17.533630 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:00:17.533630 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:00:17.534597 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:00:17.534597 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:00:17.534597 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:00:17.534597 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 17/34] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:00:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:00:17.534597 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b58d92-3a54-4c93-87e6-ba5ac6b37a28 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:17.571076 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:17.603843 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:17.649986 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af0e8b3-c287-4ba3-9063-f9fa41bec895 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:17.657795 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b58d92-3a54-4c93-87e6-ba5ac6b37a28 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:17.696809 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c0fc5d-9203-495f-baa4-083e8d75e053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:17.697516 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c0fc5d-9203-495f-baa4-083e8d75e053 None admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:17.697893 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c0fc5d-9203-495f-baa4-083e8d75e053 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:17.698229 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c0fc5d-9203-495f-baa4-083e8d75e053 None admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:17.699017 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c0fc5d-9203-495f-baa4-083e8d75e053 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:17.699752 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcecf43b-32b1-4f2d-bc0f-cfa7e8a87b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:17.700511 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcecf43b-32b1-4f2d-bc0f-cfa7e8a87b95 None admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:17.700669 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c0fc5d-9203-495f-baa4-083e8d75e053 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:17.701084 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcecf43b-32b1-4f2d-bc0f-cfa7e8a87b95 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:17.701522 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcecf43b-32b1-4f2d-bc0f-cfa7e8a87b95 None admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:17.702332 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcecf43b-32b1-4f2d-bc0f-cfa7e8a87b95 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:17.707597 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcecf43b-32b1-4f2d-bc0f-cfa7e8a87b95 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:17.845016 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 18/35] 10.222.0.61 () {60 vars in 1131 bytes} [Fri Aug 26 12:00:17 2022] POST /identity/v3/roles => generated 219 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:00:17.846306 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcecf43b-32b1-4f2d-bc0f-cfa7e8a87b95 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:17.848322 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 18/36] 10.222.0.61 () {60 vars in 1131 bytes} [Fri Aug 26 12:00:17 2022] POST /identity/v3/roles => generated 225 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:00:17.849345 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c0fc5d-9203-495f-baa4-083e8d75e053 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:17.947237 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:17.969705 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcecf43b-32b1-4f2d-bc0f-cfa7e8a87b95 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:18.019786 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74e414a-7029-4189-b715-5a9be337aafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:18.020703 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e414a-7029-4189-b715-5a9be337aafe None admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:18.020703 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e414a-7029-4189-b715-5a9be337aafe None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:18.020871 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e414a-7029-4189-b715-5a9be337aafe None admin] PATH_INFO: `/v3/domains/default/users/d101ea12e0cd40e0b25022bf649ac734/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:18.025529 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74e414a-7029-4189-b715-5a9be337aafe None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=d101ea12e0cd40e0b25022bf649ac734, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:18.027096 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74e414a-7029-4189-b715-5a9be337aafe None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:18.051544 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c0fc5d-9203-495f-baa4-083e8d75e053 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:18.055708 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 19/37] 10.222.0.61 () {58 vars in 1361 bytes} [Fri Aug 26 12:00:17 2022] PUT /identity/v3/domains/default/users/d101ea12e0cd40e0b25022bf649ac734/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:00:18.057070 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3583cbce-32f5-4716-8a75-96b95604ebb9 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:18.057356 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3583cbce-32f5-4716-8a75-96b95604ebb9 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:18.057630 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3583cbce-32f5-4716-8a75-96b95604ebb9 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:18.059319 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 20/38] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:00:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:00:18.065585 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c2256a-cda0-42bc-b0e8-d7d070fa56fd None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:18.065878 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c2256a-cda0-42bc-b0e8-d7d070fa56fd None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:18.066151 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c2256a-cda0-42bc-b0e8-d7d070fa56fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:18.087085 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-97c2256a-cda0-42bc-b0e8-d7d070fa56fd None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:00:18.088678 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-97c2256a-cda0-42bc-b0e8-d7d070fa56fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:00:18.096091 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:18.104352 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50021f54-dca8-43be-9efa-28867a2a297b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:18.105549 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50021f54-dca8-43be-9efa-28867a2a297b None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:18.106067 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50021f54-dca8-43be-9efa-28867a2a297b None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:18.106545 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50021f54-dca8-43be-9efa-28867a2a297b None admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:18.107478 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50021f54-dca8-43be-9efa-28867a2a297b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:00:18.108783 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50021f54-dca8-43be-9efa-28867a2a297b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:18.110844 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50021f54-dca8-43be-9efa-28867a2a297b None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:18.134544 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 19/39] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:00:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:18.136209 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0310afa7-ee36-476c-98fd-1c70530d65f9 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:18.136601 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0310afa7-ee36-476c-98fd-1c70530d65f9 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:18.137134 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0310afa7-ee36-476c-98fd-1c70530d65f9 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:18.138801 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 20/40] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:00:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:00:18.146646 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab23fb44-865b-4836-b14f-a95696d2e662 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:18.147279 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab23fb44-865b-4836-b14f-a95696d2e662 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:18.147663 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab23fb44-865b-4836-b14f-a95696d2e662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:18.169401 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-ab23fb44-865b-4836-b14f-a95696d2e662 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:00:18.171839 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-ab23fb44-865b-4836-b14f-a95696d2e662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:00:18.182971 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 21/41] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:00:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:00:18.184590 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c2256a-cda0-42bc-b0e8-d7d070fa56fd None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:18.270705 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 21/42] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:00:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:00:18.273134 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:18.310921 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c2256a-cda0-42bc-b0e8-d7d070fa56fd None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:18.349014 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525e3a00-60ef-4906-a43e-625ca14c9ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:18.349952 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e3a00-60ef-4906-a43e-625ca14c9ba9 None admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:18.350409 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e3a00-60ef-4906-a43e-625ca14c9ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:18.350857 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e3a00-60ef-4906-a43e-625ca14c9ba9 None admin] PATH_INFO: `/v3/projects` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:18.351712 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525e3a00-60ef-4906-a43e-625ca14c9ba9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:18.353358 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525e3a00-60ef-4906-a43e-625ca14c9ba9 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:18.386519 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 22/43] 10.222.0.61 () {60 vars in 1141 bytes} [Fri Aug 26 12:00:18 2022] POST /identity/v3/projects => generated 301 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:00:18.475661 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab23fb44-865b-4836-b14f-a95696d2e662 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:18.553924 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525e3a00-60ef-4906-a43e-625ca14c9ba9 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:18.592355 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab23fb44-865b-4836-b14f-a95696d2e662 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:18.618378 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:18.635447 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a73117-33f5-4047-8c4c-1d899cc8e529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:18.636058 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a73117-33f5-4047-8c4c-1d899cc8e529 None admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:18.636339 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a73117-33f5-4047-8c4c-1d899cc8e529 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:18.636617 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a73117-33f5-4047-8c4c-1d899cc8e529 None admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:18.637307 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a73117-33f5-4047-8c4c-1d899cc8e529 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:18.638700 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a73117-33f5-4047-8c4c-1d899cc8e529 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:18.653074 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 22/44] 10.222.0.61 () {60 vars in 1131 bytes} [Fri Aug 26 12:00:18 2022] POST /identity/v3/roles => generated 223 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:00:18.690722 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525e3a00-60ef-4906-a43e-625ca14c9ba9 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:18.716274 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:18.731460 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee1242-66d2-46d8-821b-c7d8793f9e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:18.732121 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee1242-66d2-46d8-821b-c7d8793f9e8d None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:18.732407 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee1242-66d2-46d8-821b-c7d8793f9e8d None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:18.732682 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee1242-66d2-46d8-821b-c7d8793f9e8d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:18.734562 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ee1242-66d2-46d8-821b-c7d8793f9e8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:18.736086 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ee1242-66d2-46d8-821b-c7d8793f9e8d None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:18.737956 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 23/45] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:00:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:00:18.746464 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a73117-33f5-4047-8c4c-1d899cc8e529 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:18.855846 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a73117-33f5-4047-8c4c-1d899cc8e529 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:18.890419 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73df186-3fd0-4d3e-926a-31d982b6ee30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:18.891039 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73df186-3fd0-4d3e-926a-31d982b6ee30 None admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:18.891356 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73df186-3fd0-4d3e-926a-31d982b6ee30 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:18.891633 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73df186-3fd0-4d3e-926a-31d982b6ee30 None admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:18.892272 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73df186-3fd0-4d3e-926a-31d982b6ee30 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:18.893721 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73df186-3fd0-4d3e-926a-31d982b6ee30 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:18.923011 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 23/46] 10.222.0.61 () {60 vars in 1141 bytes} [Fri Aug 26 12:00:18 2022] POST /identity/v3/projects => generated 312 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:00:20.100716 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af84d1b-6480-4aba-820a-23031f24c8b4 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:20.101197 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af84d1b-6480-4aba-820a-23031f24c8b4 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:20.101197 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af84d1b-6480-4aba-820a-23031f24c8b4 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:20.102514 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 24/47] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:00:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:00:20.108268 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e05e3e-8ee9-469f-8187-e22c276cac98 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:20.108859 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e05e3e-8ee9-469f-8187-e22c276cac98 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:20.108859 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e05e3e-8ee9-469f-8187-e22c276cac98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:20.232846 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-e6e05e3e-8ee9-469f-8187-e22c276cac98 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:00:20.234048 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-e6e05e3e-8ee9-469f-8187-e22c276cac98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:00:20.260413 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:20.300893 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 24/48] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:00:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:00:20.567918 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af84d1b-6480-4aba-820a-23031f24c8b4 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:20.672292 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af84d1b-6480-4aba-820a-23031f24c8b4 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:20.705949 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c74a8e6-3c35-45b5-85cb-19ee70c31ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:20.706579 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c74a8e6-3c35-45b5-85cb-19ee70c31ea1 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:20.706863 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c74a8e6-3c35-45b5-85cb-19ee70c31ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:20.707138 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c74a8e6-3c35-45b5-85cb-19ee70c31ea1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:20.719575 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c74a8e6-3c35-45b5-85cb-19ee70c31ea1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:00:20.725464 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-2c74a8e6-3c35-45b5-85cb-19ee70c31ea1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 12:00:20.725464 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:00:20.725464 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:00:20.725464 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:00:20.725464 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:00:20.725464 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:00:20.725464 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:00:20.725464 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:00:20.725464 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:00:20.725464 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:00:20.725464 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:00:20.725464 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:00:20.725464 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:00:20.725464 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:00:20.725464 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:00:20.725464 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:00:20.726504 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:00:20.726504 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:00:20.726504 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:00:20.726504 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:00:20.726504 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:00:20.726504 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:00:20.726504 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:00:20.726504 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:00:20.726504 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:00:20.726504 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:00:20.726504 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:00:20.726504 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:00:20.726504 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:00:20.726504 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:00:20.726504 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:00:20.726504 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:00:20.728383 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:00:20.728383 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:00:20.728383 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:00:20.728383 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:00:20.728383 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:00:20.728383 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:00:20.728383 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:00:20.728383 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:00:20.728383 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:00:20.728383 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:00:20.728383 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 12:00:20.728383 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:00:20.728383 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 25/49] 10.222.0.61 () {56 vars in 1096 bytes} [Fri Aug 26 12:00:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:00:20.732519 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e05e3e-8ee9-469f-8187-e22c276cac98 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:20.822085 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23d36b9-4a53-4e28-a3db-43a384182b00 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:20.822298 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23d36b9-4a53-4e28-a3db-43a384182b00 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:20.822581 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23d36b9-4a53-4e28-a3db-43a384182b00 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:20.823962 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 26/50] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:00:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 12:00:20.829627 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b69e9-30dd-4453-adb3-465a29d7d39e None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:20.829918 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b69e9-30dd-4453-adb3-465a29d7d39e None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:20.830197 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b69e9-30dd-4453-adb3-465a29d7d39e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:20.830383 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e05e3e-8ee9-469f-8187-e22c276cac98 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:20.848657 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-317b69e9-30dd-4453-adb3-465a29d7d39e None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:00:20.849846 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-317b69e9-30dd-4453-adb3-465a29d7d39e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:00:20.866244 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8fe963-5487-49b6-a2e6-bf4d4fa675e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:20.866884 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8fe963-5487-49b6-a2e6-bf4d4fa675e3 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:20.867175 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8fe963-5487-49b6-a2e6-bf4d4fa675e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:20.867480 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8fe963-5487-49b6-a2e6-bf4d4fa675e3 None admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:20.868151 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8fe963-5487-49b6-a2e6-bf4d4fa675e3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:00:20.868461 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8fe963-5487-49b6-a2e6-bf4d4fa675e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:20.869845 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8fe963-5487-49b6-a2e6-bf4d4fa675e3 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:20.878124 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 25/51] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:00:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:20.883793 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8fe963-5487-49b6-a2e6-bf4d4fa675e3 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:20.895821 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:20.910222 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:20.917598 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 27/52] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:00:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 12:00:20.983546 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8fe963-5487-49b6-a2e6-bf4d4fa675e3 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:21.017752 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f1d42c-ff51-492c-b22e-d4dba2c1db1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:21.018387 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f1d42c-ff51-492c-b22e-d4dba2c1db1b None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:21.018674 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f1d42c-ff51-492c-b22e-d4dba2c1db1b None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:21.018955 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f1d42c-ff51-492c-b22e-d4dba2c1db1b None admin] PATH_INFO: `/v3/users/d101ea12e0cd40e0b25022bf649ac734` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:21.020692 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f1d42c-ff51-492c-b22e-d4dba2c1db1b None admin] RBAC: Authorizing `identity:get_user(user_id=d101ea12e0cd40e0b25022bf649ac734)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:21.022315 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f1d42c-ff51-492c-b22e-d4dba2c1db1b None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:21.024042 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 26/53] 10.222.0.61 () {56 vars in 1177 bytes} [Fri Aug 26 12:00:20 2022] GET /identity/v3/users/d101ea12e0cd40e0b25022bf649ac734 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:00:21.030067 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317b69e9-30dd-4453-adb3-465a29d7d39e None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:21.145510 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317b69e9-30dd-4453-adb3-465a29d7d39e None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:21.183388 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2318ed-f948-468a-b72d-c56c5952ce17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:21.184020 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2318ed-f948-468a-b72d-c56c5952ce17 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:21.184376 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2318ed-f948-468a-b72d-c56c5952ce17 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:21.184648 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2318ed-f948-468a-b72d-c56c5952ce17 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:21.185753 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2318ed-f948-468a-b72d-c56c5952ce17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:21.187857 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2318ed-f948-468a-b72d-c56c5952ce17 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:21.189421 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 28/54] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:00:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:21.195251 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f1d42c-ff51-492c-b22e-d4dba2c1db1b None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:21.208353 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2318ed-f948-468a-b72d-c56c5952ce17 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:21.301903 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:21.312442 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f1d42c-ff51-492c-b22e-d4dba2c1db1b None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:21.349007 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9c0367-8f91-440e-a557-ef99ef72a71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:21.349630 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c0367-8f91-440e-a557-ef99ef72a71f None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:21.349918 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c0367-8f91-440e-a557-ef99ef72a71f None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:21.350195 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c0367-8f91-440e-a557-ef99ef72a71f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:21.350930 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9c0367-8f91-440e-a557-ef99ef72a71f None admin] RBAC: Adding query filter params (role.id=70ec1ba484714c8ca3adb8fe5845107c, scope.domain.id=default, user.id=d101ea12e0cd40e0b25022bf649ac734) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:00:21.351272 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9c0367-8f91-440e-a557-ef99ef72a71f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:21.352752 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9c0367-8f91-440e-a557-ef99ef72a71f None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:21.360057 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 27/55] 10.222.0.61 () {56 vars in 1322 bytes} [Fri Aug 26 12:00:21 2022] GET /identity/v3/role_assignments?user.id=d101ea12e0cd40e0b25022bf649ac734&scope.domain.id=default&role.id=70ec1ba484714c8ca3adb8fe5845107c => generated 530 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:21.373429 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:21.429491 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2318ed-f948-468a-b72d-c56c5952ce17 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:21.464438 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0356e38-d273-49a3-9a19-e70acf694e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:21.465087 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0356e38-d273-49a3-9a19-e70acf694e21 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:21.465373 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0356e38-d273-49a3-9a19-e70acf694e21 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:21.465649 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0356e38-d273-49a3-9a19-e70acf694e21 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:21.466784 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0356e38-d273-49a3-9a19-e70acf694e21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:21.469124 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0356e38-d273-49a3-9a19-e70acf694e21 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:21.470777 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 29/56] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:00:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:21.476459 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9c0367-8f91-440e-a557-ef99ef72a71f None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:21.578894 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9c0367-8f91-440e-a557-ef99ef72a71f None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:21.610960 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf206c7-13b0-4310-af41-08880c9a5024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:21.611589 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf206c7-13b0-4310-af41-08880c9a5024 None admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:21.611870 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf206c7-13b0-4310-af41-08880c9a5024 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:21.612145 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf206c7-13b0-4310-af41-08880c9a5024 None admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:21.612780 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf206c7-13b0-4310-af41-08880c9a5024 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:21.614218 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf206c7-13b0-4310-af41-08880c9a5024 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:21.639822 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 28/57] 10.222.0.61 () {60 vars in 1140 bytes} [Fri Aug 26 12:00:21 2022] POST /identity/v3/projects => generated 298 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:00:23.570657 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193fffeb-a00c-42c3-afee-666f54e38923 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:23.571167 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193fffeb-a00c-42c3-afee-666f54e38923 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:23.571373 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193fffeb-a00c-42c3-afee-666f54e38923 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:23.572572 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 30/58] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:00:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:00:23.578360 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0180cb6b-08c3-40ca-bcd3-503cee7efe2a None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:23.578752 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0180cb6b-08c3-40ca-bcd3-503cee7efe2a None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:23.578919 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0180cb6b-08c3-40ca-bcd3-503cee7efe2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:23.597648 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-0180cb6b-08c3-40ca-bcd3-503cee7efe2a None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:00:23.598841 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-0180cb6b-08c3-40ca-bcd3-503cee7efe2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:00:23.600787 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:23.665546 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 29/59] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:00:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:00:23.935574 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193fffeb-a00c-42c3-afee-666f54e38923 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:24.036780 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193fffeb-a00c-42c3-afee-666f54e38923 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:24.052301 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:24.070103 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadaa7f7-9e76-4aff-aa66-8f7b7d030844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:24.070718 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadaa7f7-9e76-4aff-aa66-8f7b7d030844 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:24.070998 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadaa7f7-9e76-4aff-aa66-8f7b7d030844 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:24.071303 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadaa7f7-9e76-4aff-aa66-8f7b7d030844 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:24.072400 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadaa7f7-9e76-4aff-aa66-8f7b7d030844 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:24.073821 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadaa7f7-9e76-4aff-aa66-8f7b7d030844 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:24.075387 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 31/60] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:00:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:24.081369 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0180cb6b-08c3-40ca-bcd3-503cee7efe2a None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:24.178504 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0180cb6b-08c3-40ca-bcd3-503cee7efe2a None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:24.211606 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aa7598-f748-4ba5-926d-ab24d1dbc02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:24.212154 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aa7598-f748-4ba5-926d-ab24d1dbc02e None admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:24.212439 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aa7598-f748-4ba5-926d-ab24d1dbc02e None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:24.212719 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aa7598-f748-4ba5-926d-ab24d1dbc02e None admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:24.213372 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aa7598-f748-4ba5-926d-ab24d1dbc02e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:24.214812 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aa7598-f748-4ba5-926d-ab24d1dbc02e None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:24.228570 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:24.241265 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 30/61] 10.222.0.61 () {60 vars in 1132 bytes} [Fri Aug 26 12:00:24 2022] POST /identity/v3/users => generated 275 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:00:27.168305 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515e4b16-70da-4c37-be68-f3cf79ed9f9b None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:27.171321 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515e4b16-70da-4c37-be68-f3cf79ed9f9b None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:27.171848 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515e4b16-70da-4c37-be68-f3cf79ed9f9b None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:27.173713 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 32/62] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:00:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 12:00:27.178939 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3ac79-8a8d-46e0-81de-c9fab487b476 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:27.179819 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3ac79-8a8d-46e0-81de-c9fab487b476 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:27.180294 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3ac79-8a8d-46e0-81de-c9fab487b476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:27.204253 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-8de3ac79-8a8d-46e0-81de-c9fab487b476 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:00:27.205509 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-8de3ac79-8a8d-46e0-81de-c9fab487b476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:00:27.296760 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 31/63] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:00:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:00:27.431074 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5912555f-e38c-40d5-a29e-0b6e17715344 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:27.431714 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5912555f-e38c-40d5-a29e-0b6e17715344 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:27.432193 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5912555f-e38c-40d5-a29e-0b6e17715344 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:27.433997 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 33/64] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:00:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:00:27.439157 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd7b8bf-80f4-450d-b5c7-b1bb77693cdf None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:27.439740 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd7b8bf-80f4-450d-b5c7-b1bb77693cdf None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:27.440229 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd7b8bf-80f4-450d-b5c7-b1bb77693cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:27.464278 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-1cd7b8bf-80f4-450d-b5c7-b1bb77693cdf None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:00:27.465879 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-1cd7b8bf-80f4-450d-b5c7-b1bb77693cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:00:27.558092 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 32/65] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:00:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:00:27.663144 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5912555f-e38c-40d5-a29e-0b6e17715344 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:27.694926 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d90586-d4f2-4a9c-8264-09d504a0e5c8 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:27.695545 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d90586-d4f2-4a9c-8264-09d504a0e5c8 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:27.696025 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d90586-d4f2-4a9c-8264-09d504a0e5c8 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:27.697784 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 33/66] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:00:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:00:27.700884 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f3d493-a139-4aef-b6e2-03ee925b0b80 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:27.701943 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f3d493-a139-4aef-b6e2-03ee925b0b80 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:27.702422 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f3d493-a139-4aef-b6e2-03ee925b0b80 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:27.706111 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 34/67] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:00:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:00:27.707435 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e785a-3221-4596-95ca-f3ab9951334e None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:27.707925 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e785a-3221-4596-95ca-f3ab9951334e None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:27.708458 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e785a-3221-4596-95ca-f3ab9951334e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:27.729427 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-f14e785a-3221-4596-95ca-f3ab9951334e None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:00:27.731336 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-f14e785a-3221-4596-95ca-f3ab9951334e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:00:27.804784 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:27.807723 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5912555f-e38c-40d5-a29e-0b6e17715344 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:27.832043 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 35/68] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:00:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 12:00:27.833111 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78adb96-c294-4837-b970-9a727c2e167f None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:27.833403 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78adb96-c294-4837-b970-9a727c2e167f None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:27.833688 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78adb96-c294-4837-b970-9a727c2e167f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:27.865037 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccb4050-b14d-4f4b-a151-56b7f2ab48b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:27.865999 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb4050-b14d-4f4b-a151-56b7f2ab48b3 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:27.866477 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb4050-b14d-4f4b-a151-56b7f2ab48b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:27.866943 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb4050-b14d-4f4b-a151-56b7f2ab48b3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:27.873470 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-e78adb96-c294-4837-b970-9a727c2e167f None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:00:27.876960 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-e78adb96-c294-4837-b970-9a727c2e167f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:00:27.889575 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccb4050-b14d-4f4b-a151-56b7f2ab48b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:00:27.894720 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-1ccb4050-b14d-4f4b-a151-56b7f2ab48b3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 12:00:27.894720 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:00:27.894720 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:00:27.894720 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:00:27.894720 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:00:27.894720 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:00:27.894720 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:00:27.894720 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:00:27.894720 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:00:27.894720 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:00:27.894720 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:00:27.894720 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:00:27.894720 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:00:27.894720 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:00:27.894720 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:00:27.894720 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:00:27.895817 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:00:27.895817 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:00:27.895817 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:00:27.895817 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:00:27.895817 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:00:27.895817 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:00:27.895817 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:00:27.895817 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:00:27.895817 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:00:27.895817 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:00:27.895817 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:00:27.895817 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:00:27.895817 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:00:27.895817 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:00:27.895817 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:00:27.895817 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:00:27.896777 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:00:27.896777 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:00:27.896777 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:00:27.896777 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:00:27.896777 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:00:27.896777 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:00:27.896777 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:00:27.896777 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:00:27.896777 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:00:27.896777 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:00:27.896777 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 12:00:27.896777 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:00:27.898506 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 34/69] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:00:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:00:27.900315 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc994e-9956-40a7-a8da-5ec40aeff9a6 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:27.900674 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc994e-9956-40a7-a8da-5ec40aeff9a6 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:27.901049 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc994e-9956-40a7-a8da-5ec40aeff9a6 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:27.902399 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 35/70] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:00:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:00:27.907752 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fc994e-9956-40a7-a8da-5ec40aeff9a6 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:27.958444 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:27.968652 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 36/71] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:00:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:00:27.969898 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f552d5b1-0a8c-49ba-9717-d2dcd6f512f8 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:27.970211 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f552d5b1-0a8c-49ba-9717-d2dcd6f512f8 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:27.970453 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f552d5b1-0a8c-49ba-9717-d2dcd6f512f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:27.987806 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-f552d5b1-0a8c-49ba-9717-d2dcd6f512f8 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:00:27.988976 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-f552d5b1-0a8c-49ba-9717-d2dcd6f512f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:00:28.039241 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fc994e-9956-40a7-a8da-5ec40aeff9a6 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:28.060418 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 37/72] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:00:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:00:28.062042 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f552d5b1-0a8c-49ba-9717-d2dcd6f512f8 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:28.078697 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ec1a36-644c-4449-a3b1-fd92d1d9ab0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:28.079658 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ec1a36-644c-4449-a3b1-fd92d1d9ab0f None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:28.080165 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ec1a36-644c-4449-a3b1-fd92d1d9ab0f None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:28.080633 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ec1a36-644c-4449-a3b1-fd92d1d9ab0f None admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:28.081515 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ec1a36-644c-4449-a3b1-fd92d1d9ab0f None admin] RBAC: Adding query filter params (name=member) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:00:28.081989 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ec1a36-644c-4449-a3b1-fd92d1d9ab0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:28.083636 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ec1a36-644c-4449-a3b1-fd92d1d9ab0f None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:28.093517 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 36/73] 10.222.0.61 () {56 vars in 1101 bytes} [Fri Aug 26 12:00:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:28.098574 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ec1a36-644c-4449-a3b1-fd92d1d9ab0f None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:28.132179 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:28.192440 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f552d5b1-0a8c-49ba-9717-d2dcd6f512f8 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:28.214779 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ec1a36-644c-4449-a3b1-fd92d1d9ab0f None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:28.222544 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:28.230876 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe57dd4-53f9-4b97-a5fa-7cda98788d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:28.231806 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe57dd4-53f9-4b97-a5fa-7cda98788d8d None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:28.232286 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe57dd4-53f9-4b97-a5fa-7cda98788d8d None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:28.233387 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe57dd4-53f9-4b97-a5fa-7cda98788d8d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:28.244613 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe57dd4-53f9-4b97-a5fa-7cda98788d8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:00:28.250738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-bfe57dd4-53f9-4b97-a5fa-7cda98788d8d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 12:00:28.250738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:00:28.250738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:00:28.250738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:00:28.250738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:00:28.250738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:00:28.250738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:00:28.250738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:00:28.250738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:00:28.250738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:00:28.250738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:00:28.250738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:00:28.250738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:00:28.250738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:00:28.250738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:00:28.250738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:00:28.251631 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:00:28.251631 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:00:28.251631 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:00:28.251631 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:00:28.251631 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:00:28.251631 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:00:28.251631 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:00:28.251631 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:00:28.251631 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:00:28.251631 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:00:28.251631 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:00:28.251631 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:00:28.251631 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:00:28.251631 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:00:28.251631 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:00:28.251631 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:00:28.252498 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:00:28.252498 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:00:28.252498 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:00:28.252498 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:00:28.252498 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:00:28.252498 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:00:28.252498 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:00:28.252498 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:00:28.252498 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:00:28.252498 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:00:28.252498 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 12:00:28.252498 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:00:28.252498 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 38/74] 10.222.0.61 () {56 vars in 1096 bytes} [Fri Aug 26 12:00:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:00:28.253684 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe57dd4-53f9-4b97-a5fa-7cda98788d8d None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:28.255913 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cdf388-ce89-41ab-b705-2602d0008730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:28.256561 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cdf388-ce89-41ab-b705-2602d0008730 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:28.256847 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cdf388-ce89-41ab-b705-2602d0008730 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:28.257126 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cdf388-ce89-41ab-b705-2602d0008730 None admin] PATH_INFO: `/v3/users/06c7a037a0fa485aa111a5e7e792882f` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:28.273815 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cdf388-ce89-41ab-b705-2602d0008730 None admin] RBAC: Authorizing `identity:get_user(user_id=06c7a037a0fa485aa111a5e7e792882f)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:28.275329 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cdf388-ce89-41ab-b705-2602d0008730 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:28.277154 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 37/75] 10.222.0.61 () {56 vars in 1177 bytes} [Fri Aug 26 12:00:28 2022] GET /identity/v3/users/06c7a037a0fa485aa111a5e7e792882f => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:28.277763 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cdf388-ce89-41ab-b705-2602d0008730 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:28.361253 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe57dd4-53f9-4b97-a5fa-7cda98788d8d None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:28.363558 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:28.386352 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cdf388-ce89-41ab-b705-2602d0008730 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:28.395994 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f0c6ff-676f-4595-a9d8-b83f772535c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:28.396628 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f0c6ff-676f-4595-a9d8-b83f772535c0 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:28.396910 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f0c6ff-676f-4595-a9d8-b83f772535c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:28.397186 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f0c6ff-676f-4595-a9d8-b83f772535c0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:28.407824 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f0c6ff-676f-4595-a9d8-b83f772535c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:00:28.412564 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-b1f0c6ff-676f-4595-a9d8-b83f772535c0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 12:00:28.412564 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:00:28.412564 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:00:28.412564 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:00:28.412564 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:00:28.412564 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:00:28.412564 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:00:28.412564 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:00:28.412564 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:00:28.412564 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:00:28.412564 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:00:28.412564 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:00:28.412564 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:00:28.412564 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:00:28.412564 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:00:28.412564 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:00:28.413406 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:00:28.413406 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:00:28.413406 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:00:28.413406 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:00:28.413406 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:00:28.413406 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:00:28.413406 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:00:28.413406 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:00:28.413406 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:00:28.413406 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:00:28.413406 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:00:28.413406 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:00:28.413406 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:00:28.413406 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:00:28.413406 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:00:28.413406 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:00:28.414172 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:00:28.414172 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:00:28.414172 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:00:28.414172 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:00:28.414172 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:00:28.414172 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:00:28.414172 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:00:28.414172 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:00:28.414172 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:00:28.414172 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:00:28.414172 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 12:00:28.414172 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:00:28.414172 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 39/76] 10.222.0.61 () {56 vars in 1114 bytes} [Fri Aug 26 12:00:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:00:28.419119 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f0c6ff-676f-4595-a9d8-b83f772535c0 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:28.420923 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e001d5a6-e5ab-4de3-a6c4-96024880c8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:28.421554 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e001d5a6-e5ab-4de3-a6c4-96024880c8f8 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:28.421823 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e001d5a6-e5ab-4de3-a6c4-96024880c8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:28.422109 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e001d5a6-e5ab-4de3-a6c4-96024880c8f8 None admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:28.422767 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e001d5a6-e5ab-4de3-a6c4-96024880c8f8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:00:28.423070 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e001d5a6-e5ab-4de3-a6c4-96024880c8f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:28.424470 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e001d5a6-e5ab-4de3-a6c4-96024880c8f8 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:28.432564 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 38/77] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:00:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:28.433354 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e001d5a6-e5ab-4de3-a6c4-96024880c8f8 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:28.460448 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:28.522162 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f0c6ff-676f-4595-a9d8-b83f772535c0 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:28.534046 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e001d5a6-e5ab-4de3-a6c4-96024880c8f8 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:28.554730 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74248c66-60be-4013-9d41-fc02a03fb5ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:28.555379 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74248c66-60be-4013-9d41-fc02a03fb5ec None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:28.555660 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74248c66-60be-4013-9d41-fc02a03fb5ec None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:28.555932 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74248c66-60be-4013-9d41-fc02a03fb5ec None admin] PATH_INFO: `/v3/domains/default` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:28.557022 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74248c66-60be-4013-9d41-fc02a03fb5ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:28.558469 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74248c66-60be-4013-9d41-fc02a03fb5ec None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:28.560043 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 40/78] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:00:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:28.561342 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74248c66-60be-4013-9d41-fc02a03fb5ec None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:28.572243 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279db28d-4f6e-4fbe-ae77-1ac45c19295f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:28.572853 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279db28d-4f6e-4fbe-ae77-1ac45c19295f None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:28.573164 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279db28d-4f6e-4fbe-ae77-1ac45c19295f None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:28.573437 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279db28d-4f6e-4fbe-ae77-1ac45c19295f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:28.591649 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:28.640403 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279db28d-4f6e-4fbe-ae77-1ac45c19295f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:28.641947 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279db28d-4f6e-4fbe-ae77-1ac45c19295f None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:28.659901 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74248c66-60be-4013-9d41-fc02a03fb5ec None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:28.666972 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:28.696147 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230c36c9-2ba9-4720-a909-be0b353ea5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:28.696777 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c36c9-2ba9-4720-a909-be0b353ea5fb None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:28.697062 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c36c9-2ba9-4720-a909-be0b353ea5fb None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:28.697336 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c36c9-2ba9-4720-a909-be0b353ea5fb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:28.707464 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230c36c9-2ba9-4720-a909-be0b353ea5fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:00:28.712161 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-230c36c9-2ba9-4720-a909-be0b353ea5fb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 12:00:28.712161 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:00:28.712161 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:00:28.712161 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:00:28.712161 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:00:28.712161 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:00:28.712161 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:00:28.712161 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:00:28.712161 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:00:28.712161 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:00:28.712161 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:00:28.712161 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:00:28.712161 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:00:28.712161 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:00:28.712161 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:00:28.712161 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:00:28.713032 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:00:28.713032 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:00:28.713032 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:00:28.713032 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:00:28.713032 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:00:28.713032 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:00:28.713032 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:00:28.713032 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:00:28.713032 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:00:28.713032 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:00:28.713032 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:00:28.713032 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:00:28.713032 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:00:28.713032 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:00:28.713032 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:00:28.713032 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:00:28.713796 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:00:28.713796 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:00:28.713796 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:00:28.713796 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:00:28.713796 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:00:28.713796 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:00:28.713796 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:00:28.713796 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:00:28.713796 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:00:28.713796 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:00:28.713796 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 12:00:28.713796 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:00:28.713796 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 41/79] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:00:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:00:28.714426 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230c36c9-2ba9-4720-a909-be0b353ea5fb None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:28.745849 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 39/80] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:00:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 12:00:28.746532 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279db28d-4f6e-4fbe-ae77-1ac45c19295f None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:28.813281 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:28.822240 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230c36c9-2ba9-4720-a909-be0b353ea5fb None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:28.845200 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279db28d-4f6e-4fbe-ae77-1ac45c19295f None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:28.855389 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cf8eee-0a47-40d0-87b1-e2eff41da7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:28.855998 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf8eee-0a47-40d0-87b1-e2eff41da7b0 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:28.856282 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf8eee-0a47-40d0-87b1-e2eff41da7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:28.856560 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cf8eee-0a47-40d0-87b1-e2eff41da7b0 None admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:28.857229 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cf8eee-0a47-40d0-87b1-e2eff41da7b0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:00:28.857651 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cf8eee-0a47-40d0-87b1-e2eff41da7b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:28.859630 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cf8eee-0a47-40d0-87b1-e2eff41da7b0 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:28.867395 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 42/81] 10.222.0.61 () {56 vars in 1111 bytes} [Fri Aug 26 12:00:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:28.868160 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cf8eee-0a47-40d0-87b1-e2eff41da7b0 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:28.872769 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:28.882431 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be7c59d-3c4a-4530-8a8b-0ac183fd49f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:28.883352 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be7c59d-3c4a-4530-8a8b-0ac183fd49f5 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:28.883641 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be7c59d-3c4a-4530-8a8b-0ac183fd49f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:28.883916 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be7c59d-3c4a-4530-8a8b-0ac183fd49f5 None admin] PATH_INFO: `/v3/users/d101ea12e0cd40e0b25022bf649ac734` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:28.885050 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be7c59d-3c4a-4530-8a8b-0ac183fd49f5 None admin] RBAC: Authorizing `identity:get_user(user_id=d101ea12e0cd40e0b25022bf649ac734)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:28.886495 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be7c59d-3c4a-4530-8a8b-0ac183fd49f5 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:28.888113 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 40/82] 10.222.0.61 () {56 vars in 1177 bytes} [Fri Aug 26 12:00:28 2022] GET /identity/v3/users/d101ea12e0cd40e0b25022bf649ac734 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:28.889397 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be7c59d-3c4a-4530-8a8b-0ac183fd49f5 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:28.969390 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cf8eee-0a47-40d0-87b1-e2eff41da7b0 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:28.989870 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be7c59d-3c4a-4530-8a8b-0ac183fd49f5 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:29.001967 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98923da-de0e-41f2-87a6-134a7ae484a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:29.002578 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98923da-de0e-41f2-87a6-134a7ae484a9 None admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:29.002869 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98923da-de0e-41f2-87a6-134a7ae484a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:29.003146 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98923da-de0e-41f2-87a6-134a7ae484a9 None admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:29.003836 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98923da-de0e-41f2-87a6-134a7ae484a9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:29.005289 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98923da-de0e-41f2-87a6-134a7ae484a9 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:29.025503 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb39474-7279-4a6f-8147-06a039288f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:29.026111 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb39474-7279-4a6f-8147-06a039288f4e None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:29.026208 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 43/83] 10.222.0.61 () {60 vars in 1132 bytes} [Fri Aug 26 12:00:28 2022] POST /identity/v3/users => generated 289 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:00:29.026391 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb39474-7279-4a6f-8147-06a039288f4e None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:29.026667 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb39474-7279-4a6f-8147-06a039288f4e None admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:29.026941 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98923da-de0e-41f2-87a6-134a7ae484a9 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:29.027428 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb39474-7279-4a6f-8147-06a039288f4e None admin] RBAC: Adding query filter params (name=member) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:00:29.027732 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb39474-7279-4a6f-8147-06a039288f4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:29.029112 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb39474-7279-4a6f-8147-06a039288f4e None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:29.039005 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 41/84] 10.222.0.61 () {56 vars in 1101 bytes} [Fri Aug 26 12:00:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:29.041209 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb39474-7279-4a6f-8147-06a039288f4e None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:29.058948 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:29.115671 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:29.170366 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb39474-7279-4a6f-8147-06a039288f4e None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:29.176561 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98923da-de0e-41f2-87a6-134a7ae484a9 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:29.212081 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c939b5-f3aa-4a1d-a2c0-f11afcf45274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:29.212711 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c939b5-f3aa-4a1d-a2c0-f11afcf45274 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:29.213038 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c939b5-f3aa-4a1d-a2c0-f11afcf45274 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:29.213282 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c939b5-f3aa-4a1d-a2c0-f11afcf45274 None admin] PATH_INFO: `/v3/users/06c7a037a0fa485aa111a5e7e792882f` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:29.214394 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c939b5-f3aa-4a1d-a2c0-f11afcf45274 None admin] RBAC: Authorizing `identity:get_user(user_id=06c7a037a0fa485aa111a5e7e792882f)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:29.215875 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c939b5-f3aa-4a1d-a2c0-f11afcf45274 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:29.217630 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3931c9-6f91-487e-8c4c-21e36cc6cab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:29.218342 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 42/85] 10.222.0.61 () {56 vars in 1177 bytes} [Fri Aug 26 12:00:29 2022] GET /identity/v3/users/06c7a037a0fa485aa111a5e7e792882f => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:29.219117 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c939b5-f3aa-4a1d-a2c0-f11afcf45274 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:29.221856 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3931c9-6f91-487e-8c4c-21e36cc6cab9 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:29.222451 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3931c9-6f91-487e-8c4c-21e36cc6cab9 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:29.222809 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3931c9-6f91-487e-8c4c-21e36cc6cab9 None admin] PATH_INFO: `/v3/projects/f66cc7270ac04520ad4a66086d154421` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:29.224986 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3931c9-6f91-487e-8c4c-21e36cc6cab9 None admin] RBAC: Authorizing `identity:get_project(project_id=f66cc7270ac04520ad4a66086d154421)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:29.226452 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3931c9-6f91-487e-8c4c-21e36cc6cab9 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:29.228340 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 44/86] 10.222.0.61 () {56 vars in 1186 bytes} [Fri Aug 26 12:00:29 2022] GET /identity/v3/projects/f66cc7270ac04520ad4a66086d154421 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:29.229408 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3931c9-6f91-487e-8c4c-21e36cc6cab9 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:29.336331 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:29.345071 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c939b5-f3aa-4a1d-a2c0-f11afcf45274 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:29.367902 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3931c9-6f91-487e-8c4c-21e36cc6cab9 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:29.373759 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:29.391990 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cc1d05-5280-41e9-8efc-369cb21ed508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:29.393435 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cc1d05-5280-41e9-8efc-369cb21ed508 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:29.393923 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cc1d05-5280-41e9-8efc-369cb21ed508 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:29.394684 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cc1d05-5280-41e9-8efc-369cb21ed508 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:29.409690 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5d192b-9855-47f2-923f-83d9fd0b1113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:29.410725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d192b-9855-47f2-923f-83d9fd0b1113 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:29.411007 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d192b-9855-47f2-923f-83d9fd0b1113 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:29.411325 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d192b-9855-47f2-923f-83d9fd0b1113 None admin] PATH_INFO: `/v3/users/06c7a037a0fa485aa111a5e7e792882f` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:29.412495 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5d192b-9855-47f2-923f-83d9fd0b1113 None admin] RBAC: Authorizing `identity:get_user(user_id=06c7a037a0fa485aa111a5e7e792882f)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:29.414166 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5d192b-9855-47f2-923f-83d9fd0b1113 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:29.419229 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 45/87] 10.222.0.61 () {56 vars in 1177 bytes} [Fri Aug 26 12:00:29 2022] GET /identity/v3/users/06c7a037a0fa485aa111a5e7e792882f => generated 275 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:29.419229 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5d192b-9855-47f2-923f-83d9fd0b1113 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:29.475406 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cc1d05-5280-41e9-8efc-369cb21ed508 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:29.478371 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cc1d05-5280-41e9-8efc-369cb21ed508 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:29.537470 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5d192b-9855-47f2-923f-83d9fd0b1113 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:29.585877 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8621ca8-dc3e-4ec1-80de-e97256dfadda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:29.586526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8621ca8-dc3e-4ec1-80de-e97256dfadda None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:29.586814 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8621ca8-dc3e-4ec1-80de-e97256dfadda None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:29.587125 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8621ca8-dc3e-4ec1-80de-e97256dfadda None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:29.609542 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:29.619949 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 43/88] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:00:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 12:00:29.620888 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cc1d05-5280-41e9-8efc-369cb21ed508 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:29.656908 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:29.684730 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8621ca8-dc3e-4ec1-80de-e97256dfadda None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:29.686992 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8621ca8-dc3e-4ec1-80de-e97256dfadda None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:29.745807 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cc1d05-5280-41e9-8efc-369cb21ed508 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:29.786617 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e82e9a5-b588-430b-9473-46502f72f747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:29.787607 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e82e9a5-b588-430b-9473-46502f72f747 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:29.788097 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e82e9a5-b588-430b-9473-46502f72f747 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:29.788570 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e82e9a5-b588-430b-9473-46502f72f747 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:29.789042 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 46/89] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:00:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 12:00:29.789597 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8621ca8-dc3e-4ec1-80de-e97256dfadda None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:29.790252 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e82e9a5-b588-430b-9473-46502f72f747 None admin] RBAC: Adding query filter params (role.id=b2403690f5ad443aad059f5ee024f61c, scope.project.id=f66cc7270ac04520ad4a66086d154421, user.id=06c7a037a0fa485aa111a5e7e792882f) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:00:29.790763 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e82e9a5-b588-430b-9473-46502f72f747 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:29.792646 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e82e9a5-b588-430b-9473-46502f72f747 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:29.802070 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 44/90] 10.222.0.61 () {56 vars in 1374 bytes} [Fri Aug 26 12:00:29 2022] GET /identity/v3/role_assignments?user.id=06c7a037a0fa485aa111a5e7e792882f&scope.project.id=f66cc7270ac04520ad4a66086d154421&role.id=b2403690f5ad443aad059f5ee024f61c => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:29.803188 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e82e9a5-b588-430b-9473-46502f72f747 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:29.839248 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:29.897626 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8621ca8-dc3e-4ec1-80de-e97256dfadda None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:29.913033 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:29.928787 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e82e9a5-b588-430b-9473-46502f72f747 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:29.934755 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785a7915-f357-4a26-aa62-d36fbc5aab97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:29.935400 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a7915-f357-4a26-aa62-d36fbc5aab97 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:29.935806 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a7915-f357-4a26-aa62-d36fbc5aab97 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:29.937011 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a7915-f357-4a26-aa62-d36fbc5aab97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:29.970836 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f9d28c-1307-4109-aeae-373292c6afad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:29.972124 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f9d28c-1307-4109-aeae-373292c6afad None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:29.972655 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f9d28c-1307-4109-aeae-373292c6afad None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:29.973131 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f9d28c-1307-4109-aeae-373292c6afad None admin] PATH_INFO: `/v3/projects/f66cc7270ac04520ad4a66086d154421` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:29.975767 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f9d28c-1307-4109-aeae-373292c6afad None admin] RBAC: Authorizing `identity:get_project(project_id=f66cc7270ac04520ad4a66086d154421)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:29.981357 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f9d28c-1307-4109-aeae-373292c6afad None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:29.984351 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 45/91] 10.222.0.61 () {56 vars in 1186 bytes} [Fri Aug 26 12:00:29 2022] GET /identity/v3/projects/f66cc7270ac04520ad4a66086d154421 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:29.985528 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f9d28c-1307-4109-aeae-373292c6afad None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:30.017043 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785a7915-f357-4a26-aa62-d36fbc5aab97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:30.018522 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785a7915-f357-4a26-aa62-d36fbc5aab97 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:30.088317 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:30.123676 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f9d28c-1307-4109-aeae-373292c6afad None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:30.167869 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261254ea-5415-429a-9c2e-392f32387c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:30.168476 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261254ea-5415-429a-9c2e-392f32387c3c None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:30.168766 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261254ea-5415-429a-9c2e-392f32387c3c None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:30.169044 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261254ea-5415-429a-9c2e-392f32387c3c None admin] PATH_INFO: `/v3/projects/f66cc7270ac04520ad4a66086d154421` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:30.170413 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261254ea-5415-429a-9c2e-392f32387c3c None admin] RBAC: Authorizing `identity:get_project(project_id=f66cc7270ac04520ad4a66086d154421)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:30.171994 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261254ea-5415-429a-9c2e-392f32387c3c None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:30.177988 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 46/92] 10.222.0.61 () {56 vars in 1186 bytes} [Fri Aug 26 12:00:29 2022] GET /identity/v3/projects/f66cc7270ac04520ad4a66086d154421 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:30.178795 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261254ea-5415-429a-9c2e-392f32387c3c None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:30.183866 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 47/93] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:00:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 12:00:30.184719 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785a7915-f357-4a26-aa62-d36fbc5aab97 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:30.227054 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:30.313852 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261254ea-5415-429a-9c2e-392f32387c3c None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:30.351476 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41720f8-1c04-4c17-9b7d-667082c0ef98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:30.352086 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41720f8-1c04-4c17-9b7d-667082c0ef98 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:30.352363 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41720f8-1c04-4c17-9b7d-667082c0ef98 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:30.352636 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41720f8-1c04-4c17-9b7d-667082c0ef98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:30.353359 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41720f8-1c04-4c17-9b7d-667082c0ef98 None admin] RBAC: Adding query filter params (role.id=70ec1ba484714c8ca3adb8fe5845107c, scope.project.id=f66cc7270ac04520ad4a66086d154421, user.id=d101ea12e0cd40e0b25022bf649ac734) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:00:30.353670 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41720f8-1c04-4c17-9b7d-667082c0ef98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:30.355105 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41720f8-1c04-4c17-9b7d-667082c0ef98 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:30.361705 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 47/94] 10.222.0.61 () {56 vars in 1374 bytes} [Fri Aug 26 12:00:30 2022] GET /identity/v3/role_assignments?user.id=d101ea12e0cd40e0b25022bf649ac734&scope.project.id=f66cc7270ac04520ad4a66086d154421&role.id=70ec1ba484714c8ca3adb8fe5845107c => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:30.362594 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41720f8-1c04-4c17-9b7d-667082c0ef98 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:30.365215 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785a7915-f357-4a26-aa62-d36fbc5aab97 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:30.448099 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ded4d9f-95d0-4d04-94e5-32fc599c8f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:30.448648 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ded4d9f-95d0-4d04-94e5-32fc599c8f0b None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:30.448936 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ded4d9f-95d0-4d04-94e5-32fc599c8f0b None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:30.449218 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ded4d9f-95d0-4d04-94e5-32fc599c8f0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:30.449969 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ded4d9f-95d0-4d04-94e5-32fc599c8f0b None admin] RBAC: Adding query filter params (role.id=a60f381814fe4c1da324d13c8e7d5d20, scope.project.id=f66cc7270ac04520ad4a66086d154421, user.id=06c7a037a0fa485aa111a5e7e792882f) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:00:30.450280 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ded4d9f-95d0-4d04-94e5-32fc599c8f0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:30.451824 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ded4d9f-95d0-4d04-94e5-32fc599c8f0b None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:30.459014 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:30.465666 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 48/95] 10.222.0.61 () {56 vars in 1374 bytes} [Fri Aug 26 12:00:30 2022] GET /identity/v3/role_assignments?user.id=06c7a037a0fa485aa111a5e7e792882f&scope.project.id=f66cc7270ac04520ad4a66086d154421&role.id=a60f381814fe4c1da324d13c8e7d5d20 => generated 259 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:30.487356 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41720f8-1c04-4c17-9b7d-667082c0ef98 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:30.549074 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b791ecaa-c07d-48fb-b32a-5d61ec17dbf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:30.549807 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791ecaa-c07d-48fb-b32a-5d61ec17dbf7 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:30.550090 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791ecaa-c07d-48fb-b32a-5d61ec17dbf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:30.550364 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791ecaa-c07d-48fb-b32a-5d61ec17dbf7 None admin] PATH_INFO: `/v3/projects/c2765307b2fd4cf08246c1e63147f07e` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:30.551538 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b791ecaa-c07d-48fb-b32a-5d61ec17dbf7 None admin] RBAC: Authorizing `identity:get_project(project_id=c2765307b2fd4cf08246c1e63147f07e)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:30.553027 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b791ecaa-c07d-48fb-b32a-5d61ec17dbf7 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:30.554743 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 48/96] 10.222.0.61 () {56 vars in 1186 bytes} [Fri Aug 26 12:00:30 2022] GET /identity/v3/projects/c2765307b2fd4cf08246c1e63147f07e => generated 312 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:30.564466 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ded4d9f-95d0-4d04-94e5-32fc599c8f0b None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:30.593236 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:30.723960 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ded4d9f-95d0-4d04-94e5-32fc599c8f0b None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:30.784875 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d10a03-b908-44e5-a0fd-93766797001b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:30.785487 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d10a03-b908-44e5-a0fd-93766797001b None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:30.785767 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d10a03-b908-44e5-a0fd-93766797001b None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:30.786043 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d10a03-b908-44e5-a0fd-93766797001b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:30.786776 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d10a03-b908-44e5-a0fd-93766797001b None admin] RBAC: Adding query filter params (role.id=b2403690f5ad443aad059f5ee024f61c, scope.project.id=c2765307b2fd4cf08246c1e63147f07e, user.id=06c7a037a0fa485aa111a5e7e792882f) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:00:30.787085 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d10a03-b908-44e5-a0fd-93766797001b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:30.788567 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d10a03-b908-44e5-a0fd-93766797001b None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:30.796458 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 49/97] 10.222.0.61 () {56 vars in 1374 bytes} [Fri Aug 26 12:00:30 2022] GET /identity/v3/role_assignments?user.id=06c7a037a0fa485aa111a5e7e792882f&scope.project.id=c2765307b2fd4cf08246c1e63147f07e&role.id=b2403690f5ad443aad059f5ee024f61c => generated 259 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:31.543017 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0874191-8b45-4ce0-8c57-b12ba2f63cc4 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:31.543541 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0874191-8b45-4ce0-8c57-b12ba2f63cc4 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:31.543615 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0874191-8b45-4ce0-8c57-b12ba2f63cc4 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:31.544962 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 49/98] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:00:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:00:31.551439 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e1fdb0-ce49-4ed3-91bc-9463012052a2 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:31.551873 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e1fdb0-ce49-4ed3-91bc-9463012052a2 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:31.552217 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e1fdb0-ce49-4ed3-91bc-9463012052a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:31.571351 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-a5e1fdb0-ce49-4ed3-91bc-9463012052a2 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:00:31.572562 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-a5e1fdb0-ce49-4ed3-91bc-9463012052a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:00:31.647626 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 50/99] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:00:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:00:31.681247 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816292fd-6399-4883-9c6c-7aed6af907a3 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:31.681541 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816292fd-6399-4883-9c6c-7aed6af907a3 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:31.681807 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816292fd-6399-4883-9c6c-7aed6af907a3 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:31.683164 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 50/100] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:00:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:00:31.689768 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df85a9a-c67c-46af-8022-aa258b9c0314 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:31.690063 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df85a9a-c67c-46af-8022-aa258b9c0314 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:31.690346 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df85a9a-c67c-46af-8022-aa258b9c0314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:31.703453 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:31.718885 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-4df85a9a-c67c-46af-8022-aa258b9c0314 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:00:31.720116 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-4df85a9a-c67c-46af-8022-aa258b9c0314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:00:31.812737 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 51/101] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:00:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:00:32.038618 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816292fd-6399-4883-9c6c-7aed6af907a3 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:32.156296 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816292fd-6399-4883-9c6c-7aed6af907a3 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:32.191999 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08345f85-55a4-4fc5-99f7-3eb48fcc6e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:32.192639 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08345f85-55a4-4fc5-99f7-3eb48fcc6e04 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:32.192925 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08345f85-55a4-4fc5-99f7-3eb48fcc6e04 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:32.193202 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08345f85-55a4-4fc5-99f7-3eb48fcc6e04 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:32.204589 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08345f85-55a4-4fc5-99f7-3eb48fcc6e04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:00:32.209377 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-08345f85-55a4-4fc5-99f7-3eb48fcc6e04 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 12:00:32.209377 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:00:32.209377 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:00:32.209377 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:00:32.209377 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:00:32.209377 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:00:32.209377 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:00:32.209377 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:00:32.209377 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:00:32.209377 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:00:32.209377 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:00:32.209377 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:00:32.209377 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:00:32.209377 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:00:32.209377 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:00:32.209377 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:00:32.210265 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:00:32.210265 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:00:32.210265 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:00:32.210265 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:00:32.210265 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:00:32.210265 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:00:32.210265 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:00:32.210265 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:00:32.210265 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:00:32.210265 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:00:32.210265 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:00:32.210265 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:00:32.210265 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:00:32.210265 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:00:32.210265 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:00:32.210265 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:00:32.211064 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:00:32.211064 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:00:32.211064 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:00:32.211064 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:00:32.211064 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:00:32.211064 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:00:32.211064 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:00:32.211064 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:00:32.211064 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:00:32.211064 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:00:32.211064 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 12:00:32.211064 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:00:32.211064 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 51/102] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:00:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:00:32.216461 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df85a9a-c67c-46af-8022-aa258b9c0314 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:32.336158 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df85a9a-c67c-46af-8022-aa258b9c0314 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:32.392092 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18203f9-1f90-4e38-8294-5baa4f8fdf1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:32.393097 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18203f9-1f90-4e38-8294-5baa4f8fdf1a None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:32.393579 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18203f9-1f90-4e38-8294-5baa4f8fdf1a None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:32.394047 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18203f9-1f90-4e38-8294-5baa4f8fdf1a None admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:32.394917 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18203f9-1f90-4e38-8294-5baa4f8fdf1a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:00:32.395455 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18203f9-1f90-4e38-8294-5baa4f8fdf1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:32.397063 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18203f9-1f90-4e38-8294-5baa4f8fdf1a None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:32.398791 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:32.415575 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 52/103] 10.222.0.61 () {56 vars in 1101 bytes} [Fri Aug 26 12:00:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:32.417419 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08345f85-55a4-4fc5-99f7-3eb48fcc6e04 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:32.419431 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18203f9-1f90-4e38-8294-5baa4f8fdf1a None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:32.458129 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:32.615403 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18203f9-1f90-4e38-8294-5baa4f8fdf1a None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:32.653283 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2db8e3-627f-4d35-bcae-6ea92b0e3931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:32.654336 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2db8e3-627f-4d35-bcae-6ea92b0e3931 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:32.655260 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2db8e3-627f-4d35-bcae-6ea92b0e3931 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:32.655537 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2db8e3-627f-4d35-bcae-6ea92b0e3931 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:32.656760 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2db8e3-627f-4d35-bcae-6ea92b0e3931 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:32.658221 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2db8e3-627f-4d35-bcae-6ea92b0e3931 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:32.659982 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 53/104] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:00:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:00:32.665457 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2db8e3-627f-4d35-bcae-6ea92b0e3931 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:32.721395 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08345f85-55a4-4fc5-99f7-3eb48fcc6e04 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:32.738657 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:32.794127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2db8e3-627f-4d35-bcae-6ea92b0e3931 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:32.832688 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc84d74-a823-495d-98d2-b6f2913d42af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:32.833569 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc84d74-a823-495d-98d2-b6f2913d42af None admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:32.834014 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc84d74-a823-495d-98d2-b6f2913d42af None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:32.834418 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc84d74-a823-495d-98d2-b6f2913d42af None admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:32.835220 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc84d74-a823-495d-98d2-b6f2913d42af None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:32.836857 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc84d74-a823-495d-98d2-b6f2913d42af None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:32.844515 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c78e2b-65cf-4a77-9e3b-b86d78aeceef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:32.845174 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c78e2b-65cf-4a77-9e3b-b86d78aeceef None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:32.845464 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c78e2b-65cf-4a77-9e3b-b86d78aeceef None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:32.845748 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c78e2b-65cf-4a77-9e3b-b86d78aeceef None admin] PATH_INFO: `/v3/users/61f5e5a725cc424eb12ca035558089e3` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:32.867386 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c78e2b-65cf-4a77-9e3b-b86d78aeceef None admin] RBAC: Authorizing `identity:get_user(user_id=61f5e5a725cc424eb12ca035558089e3)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:32.868898 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c78e2b-65cf-4a77-9e3b-b86d78aeceef None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:32.871708 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 54/106] 10.222.0.61 () {60 vars in 1141 bytes} [Fri Aug 26 12:00:32 2022] POST /identity/v3/projects => generated 302 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 12:00:32.876212 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 52/106] 10.222.0.61 () {56 vars in 1177 bytes} [Fri Aug 26 12:00:32 2022] GET /identity/v3/users/61f5e5a725cc424eb12ca035558089e3 => generated 289 bytes in 454 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:32.880309 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc84d74-a823-495d-98d2-b6f2913d42af None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:32.968753 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff155e-f619-440b-848e-bfd0c409f99e None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:32.969535 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff155e-f619-440b-848e-bfd0c409f99e None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:32.970034 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff155e-f619-440b-848e-bfd0c409f99e None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:32.971659 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 53/107] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:00:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:00:32.980654 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e7daaa-5b86-48b5-9b78-3c6b6d5d0ca6 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:32.981285 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e7daaa-5b86-48b5-9b78-3c6b6d5d0ca6 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:32.981750 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e7daaa-5b86-48b5-9b78-3c6b6d5d0ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:33.016643 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc84d74-a823-495d-98d2-b6f2913d42af None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:33.017443 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-76e7daaa-5b86-48b5-9b78-3c6b6d5d0ca6 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:00:33.018835 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-76e7daaa-5b86-48b5-9b78-3c6b6d5d0ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:00:33.036957 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:33.062436 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6c482f-4b10-4287-8a0b-2be5e70e1699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:33.063352 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6c482f-4b10-4287-8a0b-2be5e70e1699 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:33.063769 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6c482f-4b10-4287-8a0b-2be5e70e1699 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:33.064213 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6c482f-4b10-4287-8a0b-2be5e70e1699 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:33.128254 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 54/108] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:00:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:00:33.161466 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6c482f-4b10-4287-8a0b-2be5e70e1699 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:33.163397 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6c482f-4b10-4287-8a0b-2be5e70e1699 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:33.268795 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 55/109] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:00:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 12:00:33.272088 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38d9dbc-67f8-4cdc-a3d9-68047af7d47a None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:33.272088 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38d9dbc-67f8-4cdc-a3d9-68047af7d47a None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:33.272088 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38d9dbc-67f8-4cdc-a3d9-68047af7d47a None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:33.273550 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 55/110] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:00:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:00:33.278304 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38d9dbc-67f8-4cdc-a3d9-68047af7d47a None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:33.278870 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d229ed3a-0f03-445c-b23d-4d7321572eea None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:33.280263 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d229ed3a-0f03-445c-b23d-4d7321572eea None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:33.280718 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d229ed3a-0f03-445c-b23d-4d7321572eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:33.287528 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:33.301855 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-d229ed3a-0f03-445c-b23d-4d7321572eea None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:00:33.303103 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-d229ed3a-0f03-445c-b23d-4d7321572eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:00:33.316005 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:33.380130 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 56/111] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:00:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:00:33.444356 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38d9dbc-67f8-4cdc-a3d9-68047af7d47a None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:33.484199 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b034f37-98f8-4200-b0c7-acb04491ed23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:33.484839 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b034f37-98f8-4200-b0c7-acb04491ed23 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:33.485235 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b034f37-98f8-4200-b0c7-acb04491ed23 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:33.485514 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b034f37-98f8-4200-b0c7-acb04491ed23 None admin] PATH_INFO: `/v3/projects/f66cc7270ac04520ad4a66086d154421` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:33.487804 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b034f37-98f8-4200-b0c7-acb04491ed23 None admin] RBAC: Authorizing `identity:get_project(project_id=f66cc7270ac04520ad4a66086d154421)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:33.489322 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b034f37-98f8-4200-b0c7-acb04491ed23 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:33.490960 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 56/112] 10.222.0.61 () {56 vars in 1186 bytes} [Fri Aug 26 12:00:33 2022] GET /identity/v3/projects/f66cc7270ac04520ad4a66086d154421 => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:00:33.497073 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d229ed3a-0f03-445c-b23d-4d7321572eea None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:33.615310 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d229ed3a-0f03-445c-b23d-4d7321572eea None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:33.641622 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b034f37-98f8-4200-b0c7-acb04491ed23 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:33.662822 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac82338-f9c5-4288-b402-58e60168630b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:33.663569 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac82338-f9c5-4288-b402-58e60168630b None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:33.663855 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac82338-f9c5-4288-b402-58e60168630b None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:33.664129 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac82338-f9c5-4288-b402-58e60168630b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:33.664961 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac82338-f9c5-4288-b402-58e60168630b None admin] RBAC: Adding query filter params (role.id=12a2a8379bd241a098646f1c8d881614, scope.project.id=f66cc7270ac04520ad4a66086d154421, user.id=61f5e5a725cc424eb12ca035558089e3) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:00:33.665273 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac82338-f9c5-4288-b402-58e60168630b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:33.666731 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac82338-f9c5-4288-b402-58e60168630b None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:33.676102 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 57/113] 10.222.0.61 () {56 vars in 1374 bytes} [Fri Aug 26 12:00:33 2022] GET /identity/v3/role_assignments?user.id=61f5e5a725cc424eb12ca035558089e3&scope.project.id=f66cc7270ac04520ad4a66086d154421&role.id=12a2a8379bd241a098646f1c8d881614 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:33.752816 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eabaa28-5512-4dec-8bc8-5ca761ebf1fa None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:33.753371 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eabaa28-5512-4dec-8bc8-5ca761ebf1fa None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:33.753873 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eabaa28-5512-4dec-8bc8-5ca761ebf1fa None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:33.755751 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 58/114] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:00:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:00:33.757390 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c9ce8f-5298-4fd2-bd6e-9f1f2975806d None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:33.757882 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c9ce8f-5298-4fd2-bd6e-9f1f2975806d None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:33.758603 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c9ce8f-5298-4fd2-bd6e-9f1f2975806d None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:33.760798 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 59/115] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:00:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:00:33.762944 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0158b18a-7366-4154-99e8-a757869efeb0 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:33.767780 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0158b18a-7366-4154-99e8-a757869efeb0 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:33.768382 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0158b18a-7366-4154-99e8-a757869efeb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:33.792358 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-0158b18a-7366-4154-99e8-a757869efeb0 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:00:33.793938 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-0158b18a-7366-4154-99e8-a757869efeb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:00:33.798383 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:33.801497 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:33.854398 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b034f37-98f8-4200-b0c7-acb04491ed23 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:33.894487 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3653d920-f783-43ac-b90b-3da718c3ff15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:33.895052 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3653d920-f783-43ac-b90b-3da718c3ff15 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:33.895383 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3653d920-f783-43ac-b90b-3da718c3ff15 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:33.895895 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3653d920-f783-43ac-b90b-3da718c3ff15 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:33.906449 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 60/116] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:00:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:00:33.908018 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1001a81-cf7a-4682-9a76-6f9ee1a2bb61 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:33.908503 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1001a81-cf7a-4682-9a76-6f9ee1a2bb61 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:33.908994 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1001a81-cf7a-4682-9a76-6f9ee1a2bb61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:33.917539 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3653d920-f783-43ac-b90b-3da718c3ff15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:00:33.924027 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-3653d920-f783-43ac-b90b-3da718c3ff15 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 12:00:33.924027 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:00:33.924027 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:00:33.924027 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:00:33.924027 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:00:33.924027 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:00:33.924027 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:00:33.924027 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:00:33.924027 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:00:33.924027 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:00:33.924027 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:00:33.924027 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:00:33.924027 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:00:33.924027 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:00:33.924027 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:00:33.924027 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:00:33.924893 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:00:33.924893 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:00:33.924893 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:00:33.924893 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:00:33.924893 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:00:33.924893 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:00:33.924893 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:00:33.924893 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:00:33.924893 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:00:33.924893 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:00:33.924893 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:00:33.924893 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:00:33.924893 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:00:33.924893 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:00:33.924893 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:00:33.924893 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:00:33.925654 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:00:33.925654 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:00:33.925654 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:00:33.925654 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:00:33.925654 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:00:33.925654 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:00:33.925654 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:00:33.925654 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:00:33.925654 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:00:33.925654 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:00:33.925654 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 12:00:33.925654 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:00:33.925654 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 57/117] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:00:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 12:00:33.930820 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-a1001a81-cf7a-4682-9a76-6f9ee1a2bb61 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:00:33.931851 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3653d920-f783-43ac-b90b-3da718c3ff15 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:33.932483 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-a1001a81-cf7a-4682-9a76-6f9ee1a2bb61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:00:34.055498 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 61/118] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:00:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:00:34.056715 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1001a81-cf7a-4682-9a76-6f9ee1a2bb61 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:34.070549 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3653d920-f783-43ac-b90b-3da718c3ff15 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:34.108239 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:34.109534 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba85b40-d4a6-461a-8526-df776e11c257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:34.110578 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba85b40-d4a6-461a-8526-df776e11c257 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:34.110878 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba85b40-d4a6-461a-8526-df776e11c257 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:34.111293 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba85b40-d4a6-461a-8526-df776e11c257 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:34.130963 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba85b40-d4a6-461a-8526-df776e11c257 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:00:34.136370 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-fba85b40-d4a6-461a-8526-df776e11c257 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 12:00:34.136370 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:00:34.136370 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:00:34.136370 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:00:34.136370 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:00:34.136370 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:00:34.136370 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:00:34.136370 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:00:34.136370 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:00:34.136370 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:00:34.136370 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:00:34.136370 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:00:34.136370 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:00:34.136370 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:00:34.136370 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:00:34.136370 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:00:34.137225 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:00:34.137225 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:00:34.137225 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:00:34.137225 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:00:34.137225 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:00:34.137225 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:00:34.137225 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:00:34.137225 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:00:34.137225 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:00:34.137225 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:00:34.137225 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:00:34.137225 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:00:34.137225 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:00:34.137225 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:00:34.137225 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:00:34.137225 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:00:34.137999 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:00:34.137999 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:00:34.137999 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:00:34.137999 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:00:34.137999 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:00:34.137999 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:00:34.137999 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:00:34.137999 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:00:34.137999 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:00:34.137999 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:00:34.137999 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 12:00:34.137999 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:00:34.137999 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 58/119] 10.222.0.61 () {56 vars in 1096 bytes} [Fri Aug 26 12:00:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:00:34.149062 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba85b40-d4a6-461a-8526-df776e11c257 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:34.227556 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:34.270451 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1001a81-cf7a-4682-9a76-6f9ee1a2bb61 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:34.344065 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba85b40-d4a6-461a-8526-df776e11c257 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:34.346056 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9667506d-0d96-412e-b7a5-0b683b564ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:34.346726 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667506d-0d96-412e-b7a5-0b683b564ccb None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:34.347012 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667506d-0d96-412e-b7a5-0b683b564ccb None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:34.347312 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667506d-0d96-412e-b7a5-0b683b564ccb None admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:34.347989 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9667506d-0d96-412e-b7a5-0b683b564ccb None admin] RBAC: Adding query filter params (name=member) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:00:34.348293 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9667506d-0d96-412e-b7a5-0b683b564ccb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:34.349687 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9667506d-0d96-412e-b7a5-0b683b564ccb None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:34.368008 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 62/120] 10.222.0.61 () {56 vars in 1101 bytes} [Fri Aug 26 12:00:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:34.368889 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9667506d-0d96-412e-b7a5-0b683b564ccb None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:34.410260 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d6d94d-5523-4dec-93b5-27bfe3588738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:34.410788 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d6d94d-5523-4dec-93b5-27bfe3588738 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:34.411119 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d6d94d-5523-4dec-93b5-27bfe3588738 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:34.411438 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d6d94d-5523-4dec-93b5-27bfe3588738 None admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:34.412373 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d6d94d-5523-4dec-93b5-27bfe3588738 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:00:34.412690 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d6d94d-5523-4dec-93b5-27bfe3588738 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:34.414086 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d6d94d-5523-4dec-93b5-27bfe3588738 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:34.423791 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 59/121] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:00:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:34.424457 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d6d94d-5523-4dec-93b5-27bfe3588738 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:34.489562 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:34.544518 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d6d94d-5523-4dec-93b5-27bfe3588738 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:34.565613 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9667506d-0d96-412e-b7a5-0b683b564ccb None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:34.591326 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0954706a-1ce5-4d3e-b080-1d7ab02b24a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:34.592951 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0954706a-1ce5-4d3e-b080-1d7ab02b24a4 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:34.593375 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0954706a-1ce5-4d3e-b080-1d7ab02b24a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:34.593785 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0954706a-1ce5-4d3e-b080-1d7ab02b24a4 None admin] PATH_INFO: `/v3/users/06c7a037a0fa485aa111a5e7e792882f` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:34.595155 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0954706a-1ce5-4d3e-b080-1d7ab02b24a4 None admin] RBAC: Authorizing `identity:get_user(user_id=06c7a037a0fa485aa111a5e7e792882f)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:34.595905 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:34.598330 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0954706a-1ce5-4d3e-b080-1d7ab02b24a4 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:34.600261 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 60/122] 10.222.0.61 () {56 vars in 1177 bytes} [Fri Aug 26 12:00:34 2022] GET /identity/v3/users/06c7a037a0fa485aa111a5e7e792882f => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:34.601354 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b4dd02-123a-4ecf-9f07-519772035952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:34.602069 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b4dd02-123a-4ecf-9f07-519772035952 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:34.602424 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b4dd02-123a-4ecf-9f07-519772035952 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:34.602762 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b4dd02-123a-4ecf-9f07-519772035952 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:34.606688 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0954706a-1ce5-4d3e-b080-1d7ab02b24a4 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:34.614910 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b4dd02-123a-4ecf-9f07-519772035952 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:00:34.620246 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-60b4dd02-123a-4ecf-9f07-519772035952 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 12:00:34.620246 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:00:34.620246 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:00:34.620246 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:00:34.620246 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:00:34.620246 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:00:34.620246 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:00:34.620246 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:00:34.620246 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:00:34.620246 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:00:34.620246 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:00:34.620246 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:00:34.620246 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:00:34.620246 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:00:34.620246 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:00:34.620246 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:00:34.621116 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:00:34.621116 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:00:34.621116 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:00:34.621116 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:00:34.621116 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:00:34.621116 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:00:34.621116 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:00:34.621116 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:00:34.621116 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:00:34.621116 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:00:34.621116 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:00:34.621116 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:00:34.621116 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:00:34.621116 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:00:34.621116 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:00:34.621116 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:00:34.622902 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:00:34.622902 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:00:34.622902 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:00:34.622902 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:00:34.622902 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:00:34.622902 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:00:34.622902 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:00:34.622902 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:00:34.622902 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:00:34.622902 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:00:34.622902 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 12:00:34.622902 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:00:34.622902 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 63/123] 10.222.0.61 () {56 vars in 1114 bytes} [Fri Aug 26 12:00:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:00:34.622902 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b4dd02-123a-4ecf-9f07-519772035952 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:34.744720 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0954706a-1ce5-4d3e-b080-1d7ab02b24a4 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:34.752540 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b4dd02-123a-4ecf-9f07-519772035952 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:34.778092 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:34.794597 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1d7b8b-bd00-45d0-bb92-50b14259b2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:34.796263 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1d7b8b-bd00-45d0-bb92-50b14259b2c0 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:34.796741 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1d7b8b-bd00-45d0-bb92-50b14259b2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:34.797203 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1d7b8b-bd00-45d0-bb92-50b14259b2c0 None admin] PATH_INFO: `/v3/users/d101ea12e0cd40e0b25022bf649ac734` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:34.797530 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e10077-d871-4dee-876d-d104a6b04ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:34.798170 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e10077-d871-4dee-876d-d104a6b04ae6 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:34.798461 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e10077-d871-4dee-876d-d104a6b04ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:34.798741 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e10077-d871-4dee-876d-d104a6b04ae6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:34.799385 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1d7b8b-bd00-45d0-bb92-50b14259b2c0 None admin] RBAC: Authorizing `identity:get_user(user_id=d101ea12e0cd40e0b25022bf649ac734)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:34.801656 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1d7b8b-bd00-45d0-bb92-50b14259b2c0 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:34.804218 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 61/124] 10.222.0.61 () {56 vars in 1177 bytes} [Fri Aug 26 12:00:34 2022] GET /identity/v3/users/d101ea12e0cd40e0b25022bf649ac734 => generated 247 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:34.809051 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1d7b8b-bd00-45d0-bb92-50b14259b2c0 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:34.816279 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e10077-d871-4dee-876d-d104a6b04ae6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:00:34.822038 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-82e10077-d871-4dee-876d-d104a6b04ae6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 12:00:34.822038 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:00:34.822038 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:00:34.822038 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:00:34.822038 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:00:34.822038 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:00:34.822038 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:00:34.822038 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:00:34.822038 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:00:34.822038 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:00:34.822038 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:00:34.822038 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:00:34.822038 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:00:34.822038 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:00:34.822038 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:00:34.822038 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:00:34.822888 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:00:34.822888 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:00:34.822888 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:00:34.822888 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:00:34.822888 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:00:34.822888 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:00:34.822888 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:00:34.822888 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:00:34.822888 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:00:34.822888 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:00:34.822888 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:00:34.822888 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:00:34.822888 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:00:34.822888 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:00:34.822888 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:00:34.822888 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:00:34.823705 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:00:34.823705 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:00:34.823705 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:00:34.823705 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:00:34.823705 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:00:34.823705 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:00:34.823705 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:00:34.823705 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:00:34.823705 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:00:34.823705 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:00:34.823705 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 12:00:34.823705 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:00:34.823705 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 64/125] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:00:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:00:34.824328 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e10077-d871-4dee-876d-d104a6b04ae6 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:34.906143 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:34.941515 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1d7b8b-bd00-45d0-bb92-50b14259b2c0 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:34.961370 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e10077-d871-4dee-876d-d104a6b04ae6 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:34.988614 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6b103c-7872-448e-9b78-77ecfe569430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:34.989411 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b103c-7872-448e-9b78-77ecfe569430 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:34.989847 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b103c-7872-448e-9b78-77ecfe569430 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:34.990137 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b103c-7872-448e-9b78-77ecfe569430 None admin] PATH_INFO: `/v3/projects/f66cc7270ac04520ad4a66086d154421` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:34.991602 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6b103c-7872-448e-9b78-77ecfe569430 None admin] RBAC: Authorizing `identity:get_project(project_id=f66cc7270ac04520ad4a66086d154421)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:34.993780 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6b103c-7872-448e-9b78-77ecfe569430 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:35.000942 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 62/126] 10.222.0.61 () {56 vars in 1186 bytes} [Fri Aug 26 12:00:34 2022] GET /identity/v3/projects/f66cc7270ac04520ad4a66086d154421 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:35.000942 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6b103c-7872-448e-9b78-77ecfe569430 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:35.008819 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caea3424-1127-4ee1-a6ae-6eba7ab3711f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:35.009691 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caea3424-1127-4ee1-a6ae-6eba7ab3711f None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:35.010172 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caea3424-1127-4ee1-a6ae-6eba7ab3711f None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:35.011164 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caea3424-1127-4ee1-a6ae-6eba7ab3711f None admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:35.012109 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caea3424-1127-4ee1-a6ae-6eba7ab3711f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:00:35.013097 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caea3424-1127-4ee1-a6ae-6eba7ab3711f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:35.015409 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caea3424-1127-4ee1-a6ae-6eba7ab3711f None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:35.026573 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 65/127] 10.222.0.61 () {56 vars in 1111 bytes} [Fri Aug 26 12:00:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:35.031701 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caea3424-1127-4ee1-a6ae-6eba7ab3711f None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:35.074405 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:35.124580 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6b103c-7872-448e-9b78-77ecfe569430 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:35.131956 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caea3424-1127-4ee1-a6ae-6eba7ab3711f None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:35.146623 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:35.167228 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e598e525-3d43-4498-ac3d-a90585022677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:35.168356 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598e525-3d43-4498-ac3d-a90585022677 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:35.168874 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f381ebb5-9db8-4ecc-aae5-34367159bd42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:35.169156 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598e525-3d43-4498-ac3d-a90585022677 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:35.169487 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f381ebb5-9db8-4ecc-aae5-34367159bd42 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:35.169773 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f381ebb5-9db8-4ecc-aae5-34367159bd42 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:35.170078 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f381ebb5-9db8-4ecc-aae5-34367159bd42 None admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:35.170264 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598e525-3d43-4498-ac3d-a90585022677 None admin] PATH_INFO: `/v3/projects/f66cc7270ac04520ad4a66086d154421` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:35.170749 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f381ebb5-9db8-4ecc-aae5-34367159bd42 None admin] RBAC: Adding query filter params (name=member) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:00:35.171050 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f381ebb5-9db8-4ecc-aae5-34367159bd42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:35.172138 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e598e525-3d43-4498-ac3d-a90585022677 None admin] RBAC: Authorizing `identity:get_project(project_id=f66cc7270ac04520ad4a66086d154421)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:35.172473 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f381ebb5-9db8-4ecc-aae5-34367159bd42 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:35.174289 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e598e525-3d43-4498-ac3d-a90585022677 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:35.176321 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 63/128] 10.222.0.61 () {56 vars in 1186 bytes} [Fri Aug 26 12:00:34 2022] GET /identity/v3/projects/f66cc7270ac04520ad4a66086d154421 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:35.177278 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e598e525-3d43-4498-ac3d-a90585022677 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:35.181245 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 66/129] 10.222.0.61 () {56 vars in 1101 bytes} [Fri Aug 26 12:00:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:35.185784 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f381ebb5-9db8-4ecc-aae5-34367159bd42 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:35.286453 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f381ebb5-9db8-4ecc-aae5-34367159bd42 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:35.299826 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e598e525-3d43-4498-ac3d-a90585022677 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:35.320782 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3000f8c-030e-47fe-886a-5f8fd60bad9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:35.321473 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3000f8c-030e-47fe-886a-5f8fd60bad9c None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:35.321764 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3000f8c-030e-47fe-886a-5f8fd60bad9c None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:35.322044 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3000f8c-030e-47fe-886a-5f8fd60bad9c None admin] PATH_INFO: `/v3/users/06c7a037a0fa485aa111a5e7e792882f` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:35.323193 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3000f8c-030e-47fe-886a-5f8fd60bad9c None admin] RBAC: Authorizing `identity:get_user(user_id=06c7a037a0fa485aa111a5e7e792882f)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:35.324671 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3000f8c-030e-47fe-886a-5f8fd60bad9c None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:35.331295 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 67/130] 10.222.0.61 () {56 vars in 1177 bytes} [Fri Aug 26 12:00:35 2022] GET /identity/v3/users/06c7a037a0fa485aa111a5e7e792882f => generated 275 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:35.331295 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3000f8c-030e-47fe-886a-5f8fd60bad9c None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:35.336327 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:35.347739 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f8bad5-fefd-4e47-8033-270bad0c4289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:35.348766 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f8bad5-fefd-4e47-8033-270bad0c4289 None admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:35.349250 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f8bad5-fefd-4e47-8033-270bad0c4289 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:35.349726 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f8bad5-fefd-4e47-8033-270bad0c4289 None admin] PATH_INFO: `/v3/projects/f66cc7270ac04520ad4a66086d154421/users/06c7a037a0fa485aa111a5e7e792882f/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:35.352096 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f8bad5-fefd-4e47-8033-270bad0c4289 None admin] RBAC: Authorizing `identity:create_grant(project_id=f66cc7270ac04520ad4a66086d154421, user_id=06c7a037a0fa485aa111a5e7e792882f, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:35.353918 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f8bad5-fefd-4e47-8033-270bad0c4289 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:35.379354 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 64/131] 10.222.0.61 () {58 vars in 1439 bytes} [Fri Aug 26 12:00:35 2022] PUT /identity/v3/projects/f66cc7270ac04520ad4a66086d154421/users/06c7a037a0fa485aa111a5e7e792882f/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:00:35.380073 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:35.381953 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f8bad5-fefd-4e47-8033-270bad0c4289 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:35.449931 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3000f8c-030e-47fe-886a-5f8fd60bad9c None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:35.494055 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc42ef7a-6bba-40a8-87af-6c2f751fc91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:35.494906 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc42ef7a-6bba-40a8-87af-6c2f751fc91d None admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:35.495291 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc42ef7a-6bba-40a8-87af-6c2f751fc91d None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:35.495684 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc42ef7a-6bba-40a8-87af-6c2f751fc91d None admin] PATH_INFO: `/v3/projects/f66cc7270ac04520ad4a66086d154421/users/d101ea12e0cd40e0b25022bf649ac734/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:35.497796 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc42ef7a-6bba-40a8-87af-6c2f751fc91d None admin] RBAC: Authorizing `identity:create_grant(project_id=f66cc7270ac04520ad4a66086d154421, user_id=d101ea12e0cd40e0b25022bf649ac734, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:35.500445 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc42ef7a-6bba-40a8-87af-6c2f751fc91d None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:35.520564 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 68/132] 10.222.0.61 () {58 vars in 1439 bytes} [Fri Aug 26 12:00:35 2022] PUT /identity/v3/projects/f66cc7270ac04520ad4a66086d154421/users/d101ea12e0cd40e0b25022bf649ac734/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:00:35.521887 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc42ef7a-6bba-40a8-87af-6c2f751fc91d None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:35.552737 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f8bad5-fefd-4e47-8033-270bad0c4289 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:35.598274 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8befd232-4ebc-49f8-aeaf-7e1b86c550a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:35.598939 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8befd232-4ebc-49f8-aeaf-7e1b86c550a6 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:35.599249 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8befd232-4ebc-49f8-aeaf-7e1b86c550a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:35.599587 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8befd232-4ebc-49f8-aeaf-7e1b86c550a6 None admin] PATH_INFO: `/v3/users/06c7a037a0fa485aa111a5e7e792882f` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:35.600939 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8befd232-4ebc-49f8-aeaf-7e1b86c550a6 None admin] RBAC: Authorizing `identity:get_user(user_id=06c7a037a0fa485aa111a5e7e792882f)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:35.602570 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8befd232-4ebc-49f8-aeaf-7e1b86c550a6 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:35.604382 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 65/133] 10.222.0.61 () {56 vars in 1177 bytes} [Fri Aug 26 12:00:35 2022] GET /identity/v3/users/06c7a037a0fa485aa111a5e7e792882f => generated 275 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:35.605808 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d564b8d5-7a58-41df-baae-95698f03404b None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:35.606096 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d564b8d5-7a58-41df-baae-95698f03404b None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:35.606372 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d564b8d5-7a58-41df-baae-95698f03404b None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:35.608590 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 66/134] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:00:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:00:35.611733 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d564b8d5-7a58-41df-baae-95698f03404b None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:35.665152 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc42ef7a-6bba-40a8-87af-6c2f751fc91d None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:35.668702 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:35.708332 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887ef8a6-8ea9-47ce-a55d-b164389aab06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:35.708986 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ef8a6-8ea9-47ce-a55d-b164389aab06 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:35.709270 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ef8a6-8ea9-47ce-a55d-b164389aab06 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:35.709551 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ef8a6-8ea9-47ce-a55d-b164389aab06 None admin] PATH_INFO: `/v3/projects/f66cc7270ac04520ad4a66086d154421` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:35.710687 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:35.710794 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887ef8a6-8ea9-47ce-a55d-b164389aab06 None admin] RBAC: Authorizing `identity:get_project(project_id=f66cc7270ac04520ad4a66086d154421)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:35.712503 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887ef8a6-8ea9-47ce-a55d-b164389aab06 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:35.714283 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 69/135] 10.222.0.61 () {56 vars in 1186 bytes} [Fri Aug 26 12:00:35 2022] GET /identity/v3/projects/f66cc7270ac04520ad4a66086d154421 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:35.715493 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fef997-995a-469b-a5b0-640daf532d93 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:35.715808 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fef997-995a-469b-a5b0-640daf532d93 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:35.716123 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fef997-995a-469b-a5b0-640daf532d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:35.735087 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-b8fef997-995a-469b-a5b0-640daf532d93 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:00:35.736448 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-b8fef997-995a-469b-a5b0-640daf532d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:00:35.790552 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d564b8d5-7a58-41df-baae-95698f03404b None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:35.821820 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 70/136] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:00:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:00:35.822685 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fef997-995a-469b-a5b0-640daf532d93 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:35.838546 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b67be91-b83c-40cd-9ac6-89e2f53e76fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:35.839187 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b67be91-b83c-40cd-9ac6-89e2f53e76fc None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:35.839501 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b67be91-b83c-40cd-9ac6-89e2f53e76fc None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:35.839777 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b67be91-b83c-40cd-9ac6-89e2f53e76fc None admin] PATH_INFO: `/v3/projects/c2765307b2fd4cf08246c1e63147f07e` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:35.840922 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b67be91-b83c-40cd-9ac6-89e2f53e76fc None admin] RBAC: Authorizing `identity:get_project(project_id=c2765307b2fd4cf08246c1e63147f07e)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:35.843460 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b67be91-b83c-40cd-9ac6-89e2f53e76fc None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:35.846055 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 67/137] 10.222.0.61 () {56 vars in 1186 bytes} [Fri Aug 26 12:00:35 2022] GET /identity/v3/projects/c2765307b2fd4cf08246c1e63147f07e => generated 312 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:35.852836 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b67be91-b83c-40cd-9ac6-89e2f53e76fc None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:35.941420 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fef997-995a-469b-a5b0-640daf532d93 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:35.943071 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:35.995645 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b67be91-b83c-40cd-9ac6-89e2f53e76fc None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:36.023220 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:36.032473 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688dc5f0-e491-4757-a910-b8f4e8956cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:36.033157 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688dc5f0-e491-4757-a910-b8f4e8956cf6 None admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:36.033436 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688dc5f0-e491-4757-a910-b8f4e8956cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:36.033705 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688dc5f0-e491-4757-a910-b8f4e8956cf6 None admin] PATH_INFO: `/v3/projects/f66cc7270ac04520ad4a66086d154421/users/06c7a037a0fa485aa111a5e7e792882f/roles/a60f381814fe4c1da324d13c8e7d5d20` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:36.036961 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915403f7-01df-4cb4-95a7-8cb1e6455526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:36.037525 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688dc5f0-e491-4757-a910-b8f4e8956cf6 None admin] RBAC: Authorizing `identity:create_grant(project_id=f66cc7270ac04520ad4a66086d154421, user_id=06c7a037a0fa485aa111a5e7e792882f, role_id=a60f381814fe4c1da324d13c8e7d5d20)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:36.037628 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915403f7-01df-4cb4-95a7-8cb1e6455526 None admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:36.037895 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915403f7-01df-4cb4-95a7-8cb1e6455526 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:36.038174 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915403f7-01df-4cb4-95a7-8cb1e6455526 None admin] PATH_INFO: `/v3/projects/c2765307b2fd4cf08246c1e63147f07e/users/06c7a037a0fa485aa111a5e7e792882f/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:36.039867 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688dc5f0-e491-4757-a910-b8f4e8956cf6 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:36.041718 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915403f7-01df-4cb4-95a7-8cb1e6455526 None admin] RBAC: Authorizing `identity:create_grant(project_id=c2765307b2fd4cf08246c1e63147f07e, user_id=06c7a037a0fa485aa111a5e7e792882f, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:36.043334 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915403f7-01df-4cb4-95a7-8cb1e6455526 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:36.071238 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 71/138] 10.222.0.61 () {58 vars in 1439 bytes} [Fri Aug 26 12:00:35 2022] PUT /identity/v3/projects/f66cc7270ac04520ad4a66086d154421/users/06c7a037a0fa485aa111a5e7e792882f/roles/a60f381814fe4c1da324d13c8e7d5d20 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:00:36.071238 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becdf3a6-fb21-43d0-bb17-6bf062a8c856 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:36.071238 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becdf3a6-fb21-43d0-bb17-6bf062a8c856 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:36.071238 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becdf3a6-fb21-43d0-bb17-6bf062a8c856 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:36.071238 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 72/139] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:00:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:00:36.079157 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 68/140] 10.222.0.61 () {58 vars in 1439 bytes} [Fri Aug 26 12:00:35 2022] PUT /identity/v3/projects/c2765307b2fd4cf08246c1e63147f07e/users/06c7a037a0fa485aa111a5e7e792882f/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:00:36.093220 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aace7f8-a0fa-4f0e-956f-2a2f342b522a None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:36.093559 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aace7f8-a0fa-4f0e-956f-2a2f342b522a None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:36.093785 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aace7f8-a0fa-4f0e-956f-2a2f342b522a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:36.121278 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-1aace7f8-a0fa-4f0e-956f-2a2f342b522a None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:00:36.122510 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-1aace7f8-a0fa-4f0e-956f-2a2f342b522a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:00:36.201719 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915403f7-01df-4cb4-95a7-8cb1e6455526 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:36.223341 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 73/141] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:00:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 12:00:36.352228 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915403f7-01df-4cb4-95a7-8cb1e6455526 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:36.396561 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d921551-8eaf-4e31-a107-88b74fd50d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:36.397560 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d921551-8eaf-4e31-a107-88b74fd50d73 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:36.398063 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d921551-8eaf-4e31-a107-88b74fd50d73 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:36.398557 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d921551-8eaf-4e31-a107-88b74fd50d73 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:36.399961 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d921551-8eaf-4e31-a107-88b74fd50d73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:36.401620 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d921551-8eaf-4e31-a107-88b74fd50d73 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:36.403885 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 69/142] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:00:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:00:36.410215 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aace7f8-a0fa-4f0e-956f-2a2f342b522a None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:36.526736 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:36.537267 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aace7f8-a0fa-4f0e-956f-2a2f342b522a None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:36.583647 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d921551-8eaf-4e31-a107-88b74fd50d73 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:36.583826 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f8a9af-a4ae-4fb5-a682-f5f4362d9615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:36.584285 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f8a9af-a4ae-4fb5-a682-f5f4362d9615 None admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:36.584560 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f8a9af-a4ae-4fb5-a682-f5f4362d9615 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:36.584832 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f8a9af-a4ae-4fb5-a682-f5f4362d9615 None admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:36.585533 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f8a9af-a4ae-4fb5-a682-f5f4362d9615 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:36.587008 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f8a9af-a4ae-4fb5-a682-f5f4362d9615 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:36.617621 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 74/143] 10.222.0.61 () {60 vars in 1132 bytes} [Fri Aug 26 12:00:36 2022] POST /identity/v3/users => generated 283 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:00:36.660740 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:36.728973 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d921551-8eaf-4e31-a107-88b74fd50d73 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:36.784959 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7c9906-f730-4dca-9b24-06b1741dfba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:36.785616 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7c9906-f730-4dca-9b24-06b1741dfba4 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:36.785902 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7c9906-f730-4dca-9b24-06b1741dfba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:36.786195 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7c9906-f730-4dca-9b24-06b1741dfba4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:36.801560 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7c9906-f730-4dca-9b24-06b1741dfba4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:00:36.807051 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-8e7c9906-f730-4dca-9b24-06b1741dfba4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 12:00:36.807051 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:00:36.807051 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:00:36.807051 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:00:36.807051 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:00:36.807051 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:00:36.807051 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:00:36.807051 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:00:36.807051 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:00:36.807051 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:00:36.807051 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:00:36.807051 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:00:36.807051 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:00:36.807051 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:00:36.807051 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:00:36.807051 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:00:36.807943 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:00:36.807943 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:00:36.807943 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:00:36.807943 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:00:36.807943 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:00:36.807943 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:00:36.807943 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:00:36.807943 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:00:36.807943 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:00:36.807943 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:00:36.807943 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:00:36.807943 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:00:36.807943 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:00:36.807943 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:00:36.807943 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:00:36.807943 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:00:36.808720 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:00:36.808720 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:00:36.808720 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:00:36.808720 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:00:36.808720 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:00:36.808720 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:00:36.808720 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:00:36.808720 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:00:36.808720 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:00:36.808720 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:00:36.808720 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 12:00:36.808720 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:00:36.808720 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 70/144] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:00:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:00:36.815894 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f8a9af-a4ae-4fb5-a682-f5f4362d9615 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:36.945842 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f8a9af-a4ae-4fb5-a682-f5f4362d9615 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:36.986314 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790eb2e6-cb97-4b1e-b803-14eef1644d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:36.986925 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790eb2e6-cb97-4b1e-b803-14eef1644d32 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:36.987250 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790eb2e6-cb97-4b1e-b803-14eef1644d32 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:36.987557 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790eb2e6-cb97-4b1e-b803-14eef1644d32 None admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:36.988278 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790eb2e6-cb97-4b1e-b803-14eef1644d32 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:00:36.988626 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790eb2e6-cb97-4b1e-b803-14eef1644d32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:36.990028 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790eb2e6-cb97-4b1e-b803-14eef1644d32 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:37.002083 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 75/145] 10.222.0.61 () {56 vars in 1101 bytes} [Fri Aug 26 12:00:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:37.007647 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7c9906-f730-4dca-9b24-06b1741dfba4 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:37.168299 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7c9906-f730-4dca-9b24-06b1741dfba4 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:37.184893 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:37.216001 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b6497-b5b8-4c27-b9b7-5cfa6776df46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:37.217577 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b6497-b5b8-4c27-b9b7-5cfa6776df46 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:37.218082 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b6497-b5b8-4c27-b9b7-5cfa6776df46 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:37.218556 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b6497-b5b8-4c27-b9b7-5cfa6776df46 None admin] PATH_INFO: `/v3/users/61f5e5a725cc424eb12ca035558089e3` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:37.219925 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3b6497-b5b8-4c27-b9b7-5cfa6776df46 None admin] RBAC: Authorizing `identity:get_user(user_id=61f5e5a725cc424eb12ca035558089e3)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:37.221582 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3b6497-b5b8-4c27-b9b7-5cfa6776df46 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:37.223419 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 71/146] 10.222.0.61 () {56 vars in 1177 bytes} [Fri Aug 26 12:00:37 2022] GET /identity/v3/users/61f5e5a725cc424eb12ca035558089e3 => generated 289 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:37.228954 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790eb2e6-cb97-4b1e-b803-14eef1644d32 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:37.255610 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:37.375143 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790eb2e6-cb97-4b1e-b803-14eef1644d32 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:37.430254 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e3baf-6019-4ea0-825c-2d14f81d5539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:37.430892 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e3baf-6019-4ea0-825c-2d14f81d5539 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:37.431172 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e3baf-6019-4ea0-825c-2d14f81d5539 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:37.431481 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e3baf-6019-4ea0-825c-2d14f81d5539 None admin] PATH_INFO: `/v3/projects/f66cc7270ac04520ad4a66086d154421` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:37.432629 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758e3baf-6019-4ea0-825c-2d14f81d5539 None admin] RBAC: Authorizing `identity:get_project(project_id=f66cc7270ac04520ad4a66086d154421)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:37.434081 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758e3baf-6019-4ea0-825c-2d14f81d5539 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:37.435893 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 76/147] 10.222.0.61 () {56 vars in 1186 bytes} [Fri Aug 26 12:00:37 2022] GET /identity/v3/projects/f66cc7270ac04520ad4a66086d154421 => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:37.441795 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b6497-b5b8-4c27-b9b7-5cfa6776df46 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:37.581784 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b6497-b5b8-4c27-b9b7-5cfa6776df46 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:37.627460 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ea68ae-3cb7-486b-9a07-a233c85d3ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:37.628327 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea68ae-3cb7-486b-9a07-a233c85d3ea8 None admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:37.628674 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea68ae-3cb7-486b-9a07-a233c85d3ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:37.628904 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea68ae-3cb7-486b-9a07-a233c85d3ea8 None admin] PATH_INFO: `/v3/projects/f66cc7270ac04520ad4a66086d154421/users/61f5e5a725cc424eb12ca035558089e3/roles/12a2a8379bd241a098646f1c8d881614` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:37.630879 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ea68ae-3cb7-486b-9a07-a233c85d3ea8 None admin] RBAC: Authorizing `identity:create_grant(project_id=f66cc7270ac04520ad4a66086d154421, user_id=61f5e5a725cc424eb12ca035558089e3, role_id=12a2a8379bd241a098646f1c8d881614)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:37.632375 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ea68ae-3cb7-486b-9a07-a233c85d3ea8 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:37.663786 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 72/148] 10.222.0.61 () {58 vars in 1439 bytes} [Fri Aug 26 12:00:37 2022] PUT /identity/v3/projects/f66cc7270ac04520ad4a66086d154421/users/61f5e5a725cc424eb12ca035558089e3/roles/12a2a8379bd241a098646f1c8d881614 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:00:38.296896 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f5744-16ee-462b-881b-9eee5f42e307 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:38.297496 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f5744-16ee-462b-881b-9eee5f42e307 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:38.297496 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f5744-16ee-462b-881b-9eee5f42e307 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:38.298861 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 77/149] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:00:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:00:38.307366 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed3509b-40ea-4725-87ba-b6e846e083ec None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:38.308270 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed3509b-40ea-4725-87ba-b6e846e083ec None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:38.308768 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed3509b-40ea-4725-87ba-b6e846e083ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:38.331016 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-4ed3509b-40ea-4725-87ba-b6e846e083ec None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:00:38.332666 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-4ed3509b-40ea-4725-87ba-b6e846e083ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:00:38.371324 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:38.425605 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 73/150] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:00:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:00:38.644814 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f361ba-5cb2-421f-b48f-5075b72fc74f None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:38.645507 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f361ba-5cb2-421f-b48f-5075b72fc74f None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:38.646010 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f361ba-5cb2-421f-b48f-5075b72fc74f None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:38.647869 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 78/151] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:00:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:00:38.652930 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8f608-0011-487a-96d7-7d43d8118d81 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:38.653474 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8f608-0011-487a-96d7-7d43d8118d81 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:38.653957 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8f608-0011-487a-96d7-7d43d8118d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:38.680231 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-d5c8f608-0011-487a-96d7-7d43d8118d81 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:00:38.681551 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-d5c8f608-0011-487a-96d7-7d43d8118d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:00:38.768735 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 74/152] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:00:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:00:39.064990 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f361ba-5cb2-421f-b48f-5075b72fc74f None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:39.168526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:39.200231 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f361ba-5cb2-421f-b48f-5075b72fc74f None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:39.243165 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb60a2d7-d297-43e1-9427-6056fede49fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:39.244188 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb60a2d7-d297-43e1-9427-6056fede49fb None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:39.244694 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb60a2d7-d297-43e1-9427-6056fede49fb None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:39.245173 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb60a2d7-d297-43e1-9427-6056fede49fb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:39.260146 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb60a2d7-d297-43e1-9427-6056fede49fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:00:39.266340 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-fb60a2d7-d297-43e1-9427-6056fede49fb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 12:00:39.266340 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:00:39.266340 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:00:39.266340 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:00:39.266340 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:00:39.266340 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:00:39.266340 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:00:39.266340 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:00:39.266340 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:00:39.266340 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:00:39.266340 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:00:39.266340 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:00:39.266340 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:00:39.266340 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:00:39.266340 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:00:39.266340 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:00:39.267284 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:00:39.267284 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:00:39.267284 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:00:39.267284 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:00:39.267284 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:00:39.267284 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:00:39.267284 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:00:39.267284 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:00:39.267284 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:00:39.267284 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:00:39.267284 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:00:39.267284 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:00:39.267284 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:00:39.267284 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:00:39.267284 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:00:39.267284 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:00:39.268230 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:00:39.268230 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:00:39.268230 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:00:39.268230 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:00:39.268230 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:00:39.268230 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:00:39.268230 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:00:39.268230 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:00:39.268230 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:00:39.268230 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:00:39.268230 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 12:00:39.268230 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:00:39.270468 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 79/153] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:00:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:00:39.276362 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c8f608-0011-487a-96d7-7d43d8118d81 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:39.290260 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb60a2d7-d297-43e1-9427-6056fede49fb None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:39.415620 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:39.417755 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c8f608-0011-487a-96d7-7d43d8118d81 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:39.461144 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957ffb22-8668-4d32-88a6-393a788026ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:39.462217 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ffb22-8668-4d32-88a6-393a788026ea None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:39.462698 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ffb22-8668-4d32-88a6-393a788026ea None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:39.463159 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ffb22-8668-4d32-88a6-393a788026ea None admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:39.464037 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957ffb22-8668-4d32-88a6-393a788026ea None admin] RBAC: Adding query filter params (name=member) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:00:39.464540 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957ffb22-8668-4d32-88a6-393a788026ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:39.466107 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957ffb22-8668-4d32-88a6-393a788026ea None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:39.476591 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 75/154] 10.222.0.61 () {56 vars in 1101 bytes} [Fri Aug 26 12:00:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:39.478200 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443c748b-c462-428c-9dd8-5de1ecd42195 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:39.478685 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443c748b-c462-428c-9dd8-5de1ecd42195 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:39.479151 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443c748b-c462-428c-9dd8-5de1ecd42195 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:39.480988 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 76/155] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:00:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:00:39.486587 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21e9e6-6f4e-42eb-8cb5-54345ac0f207 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:39.487284 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21e9e6-6f4e-42eb-8cb5-54345ac0f207 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:39.487763 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21e9e6-6f4e-42eb-8cb5-54345ac0f207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:39.509018 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-de21e9e6-6f4e-42eb-8cb5-54345ac0f207 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:00:39.510504 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-de21e9e6-6f4e-42eb-8cb5-54345ac0f207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:00:39.528757 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb60a2d7-d297-43e1-9427-6056fede49fb None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:39.602098 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 77/156] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:00:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 12:00:39.603575 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de21e9e6-6f4e-42eb-8cb5-54345ac0f207 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:39.620648 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de1e1f2-e8b0-4380-95c1-2198354b2abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:39.621561 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de1e1f2-e8b0-4380-95c1-2198354b2abe None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:39.621995 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de1e1f2-e8b0-4380-95c1-2198354b2abe None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:39.622418 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de1e1f2-e8b0-4380-95c1-2198354b2abe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:39.640323 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:39.659825 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de1e1f2-e8b0-4380-95c1-2198354b2abe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:00:39.665262 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-5de1e1f2-e8b0-4380-95c1-2198354b2abe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 12:00:39.665262 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:00:39.665262 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:00:39.665262 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:00:39.665262 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:00:39.665262 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:00:39.665262 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:00:39.665262 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:00:39.665262 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:00:39.665262 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:00:39.665262 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:00:39.665262 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:00:39.665262 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:00:39.665262 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:00:39.665262 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:00:39.665262 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:00:39.666438 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:00:39.666438 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:00:39.666438 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:00:39.666438 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:00:39.666438 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:00:39.666438 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:00:39.666438 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:00:39.666438 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:00:39.666438 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:00:39.666438 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:00:39.666438 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:00:39.666438 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:00:39.666438 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:00:39.666438 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:00:39.666438 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:00:39.666438 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:00:39.667336 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:00:39.667336 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:00:39.667336 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:00:39.667336 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:00:39.667336 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:00:39.667336 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:00:39.667336 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:00:39.667336 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:00:39.667336 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:00:39.667336 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:00:39.667336 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 12:00:39.667336 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:00:39.669794 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 80/157] 10.222.0.61 () {56 vars in 1096 bytes} [Fri Aug 26 12:00:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 12:00:39.675777 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de1e1f2-e8b0-4380-95c1-2198354b2abe None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:39.728042 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:39.735577 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de21e9e6-6f4e-42eb-8cb5-54345ac0f207 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:39.777239 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc55a4a9-789e-40b6-8329-1bde40fe93d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:39.778218 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55a4a9-789e-40b6-8329-1bde40fe93d2 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:39.778751 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55a4a9-789e-40b6-8329-1bde40fe93d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:39.779232 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55a4a9-789e-40b6-8329-1bde40fe93d2 None admin] PATH_INFO: `/v3/users/06c7a037a0fa485aa111a5e7e792882f` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:39.780604 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc55a4a9-789e-40b6-8329-1bde40fe93d2 None admin] RBAC: Authorizing `identity:get_user(user_id=06c7a037a0fa485aa111a5e7e792882f)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:39.782616 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc55a4a9-789e-40b6-8329-1bde40fe93d2 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:39.785111 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 78/158] 10.222.0.61 () {56 vars in 1177 bytes} [Fri Aug 26 12:00:39 2022] GET /identity/v3/users/06c7a037a0fa485aa111a5e7e792882f => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:39.786665 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3dbdd1-ba2a-4576-8cb1-03e6f38da962 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:39.787136 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3dbdd1-ba2a-4576-8cb1-03e6f38da962 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:39.787748 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3dbdd1-ba2a-4576-8cb1-03e6f38da962 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:39.789511 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 79/159] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:00:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:00:39.795103 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3dbdd1-ba2a-4576-8cb1-03e6f38da962 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:39.944901 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de1e1f2-e8b0-4380-95c1-2198354b2abe None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:39.953678 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3dbdd1-ba2a-4576-8cb1-03e6f38da962 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:40.006704 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a7c9db-bf44-4e60-a879-ceb45ae0ed69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:40.007712 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a7c9db-bf44-4e60-a879-ceb45ae0ed69 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:40.008452 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a7c9db-bf44-4e60-a879-ceb45ae0ed69 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:40.008956 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a7c9db-bf44-4e60-a879-ceb45ae0ed69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:40.023249 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ed8652-88e2-4827-8682-90c2ece604c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:40.023249 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ed8652-88e2-4827-8682-90c2ece604c0 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:40.023249 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ed8652-88e2-4827-8682-90c2ece604c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:40.023249 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ed8652-88e2-4827-8682-90c2ece604c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:40.023646 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ed8652-88e2-4827-8682-90c2ece604c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:00:40.023646 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ed8652-88e2-4827-8682-90c2ece604c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:40.023646 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ed8652-88e2-4827-8682-90c2ece604c0 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:40.043503 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 81/160] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:00:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:00:40.043503 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e7cf3-a3aa-43bf-94b4-6438496f14d1 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:40.043503 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e7cf3-a3aa-43bf-94b4-6438496f14d1 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:40.043503 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e7cf3-a3aa-43bf-94b4-6438496f14d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:40.055555 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:40.073074 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-f52e7cf3-a3aa-43bf-94b4-6438496f14d1 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:00:40.078956 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-f52e7cf3-a3aa-43bf-94b4-6438496f14d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:00:40.139662 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a7c9db-bf44-4e60-a879-ceb45ae0ed69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:40.141428 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a7c9db-bf44-4e60-a879-ceb45ae0ed69 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:40.184849 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:40.206651 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 82/161] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:00:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:00:40.208551 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52e7cf3-a3aa-43bf-94b4-6438496f14d1 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:40.348704 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 80/162] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:00:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 554 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 12:00:40.350002 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a7c9db-bf44-4e60-a879-ceb45ae0ed69 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:40.404051 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52e7cf3-a3aa-43bf-94b4-6438496f14d1 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:40.455556 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:40.477257 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67e8cd0-5107-4fa1-bf87-66b0af1b3f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:40.477830 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67e8cd0-5107-4fa1-bf87-66b0af1b3f4a None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:40.478110 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67e8cd0-5107-4fa1-bf87-66b0af1b3f4a None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:40.478385 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67e8cd0-5107-4fa1-bf87-66b0af1b3f4a None admin] PATH_INFO: `/v3/users/d101ea12e0cd40e0b25022bf649ac734` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:40.482254 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67e8cd0-5107-4fa1-bf87-66b0af1b3f4a None admin] RBAC: Authorizing `identity:get_user(user_id=d101ea12e0cd40e0b25022bf649ac734)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:40.484152 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67e8cd0-5107-4fa1-bf87-66b0af1b3f4a None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:40.485939 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 83/163] 10.222.0.61 () {56 vars in 1177 bytes} [Fri Aug 26 12:00:40 2022] GET /identity/v3/users/d101ea12e0cd40e0b25022bf649ac734 => generated 247 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:40.486617 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67e8cd0-5107-4fa1-bf87-66b0af1b3f4a None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:40.546577 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a7c9db-bf44-4e60-a879-ceb45ae0ed69 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:40.607086 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:40.633700 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2333aa79-27ec-40d5-b64e-bf5864413d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:40.634387 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2333aa79-27ec-40d5-b64e-bf5864413d91 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:40.634674 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2333aa79-27ec-40d5-b64e-bf5864413d91 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:40.634943 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2333aa79-27ec-40d5-b64e-bf5864413d91 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:40.668201 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2333aa79-27ec-40d5-b64e-bf5864413d91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:00:40.673924 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-2333aa79-27ec-40d5-b64e-bf5864413d91 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 12:00:40.673924 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:00:40.673924 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:00:40.673924 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:00:40.673924 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:00:40.673924 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:00:40.673924 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:00:40.673924 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:00:40.673924 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:00:40.673924 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:00:40.673924 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:00:40.673924 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:00:40.673924 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:00:40.673924 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:00:40.673924 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:00:40.673924 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:00:40.674800 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:00:40.674800 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:00:40.674800 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:00:40.674800 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:00:40.674800 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:00:40.674800 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:00:40.674800 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:00:40.674800 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:00:40.674800 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:00:40.674800 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:00:40.674800 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:00:40.674800 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:00:40.674800 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:00:40.674800 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:00:40.674800 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:00:40.674800 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:00:40.675642 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:00:40.675642 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:00:40.675642 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:00:40.675642 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:00:40.675642 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:00:40.675642 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:00:40.675642 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:00:40.675642 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:00:40.675642 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:00:40.675642 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:00:40.675642 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 12:00:40.675642 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:00:40.675642 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 81/164] 10.222.0.61 () {56 vars in 1114 bytes} [Fri Aug 26 12:00:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:00:40.676302 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2333aa79-27ec-40d5-b64e-bf5864413d91 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:40.698608 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67e8cd0-5107-4fa1-bf87-66b0af1b3f4a None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:40.792432 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf3b1b-de88-4e8d-9530-ab08f0f37ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:40.793303 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf3b1b-de88-4e8d-9530-ab08f0f37ef6 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:40.793723 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf3b1b-de88-4e8d-9530-ab08f0f37ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:40.794884 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf3b1b-de88-4e8d-9530-ab08f0f37ef6 None admin] PATH_INFO: `/v3/projects/f66cc7270ac04520ad4a66086d154421` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:40.797422 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cf3b1b-de88-4e8d-9530-ab08f0f37ef6 None admin] RBAC: Authorizing `identity:get_project(project_id=f66cc7270ac04520ad4a66086d154421)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:40.800798 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cf3b1b-de88-4e8d-9530-ab08f0f37ef6 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:40.807718 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 84/165] 10.222.0.61 () {56 vars in 1186 bytes} [Fri Aug 26 12:00:40 2022] GET /identity/v3/projects/f66cc7270ac04520ad4a66086d154421 => generated 298 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:40.809149 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf3b1b-de88-4e8d-9530-ab08f0f37ef6 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:40.838679 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2333aa79-27ec-40d5-b64e-bf5864413d91 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:40.858184 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:40.879531 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7583eb6d-c562-4890-977c-5c34e6cc1a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:40.880505 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7583eb6d-c562-4890-977c-5c34e6cc1a3d None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:40.880940 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7583eb6d-c562-4890-977c-5c34e6cc1a3d None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:40.881365 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7583eb6d-c562-4890-977c-5c34e6cc1a3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:40.942492 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf3b1b-de88-4e8d-9530-ab08f0f37ef6 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:40.966128 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7583eb6d-c562-4890-977c-5c34e6cc1a3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:40.968580 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7583eb6d-c562-4890-977c-5c34e6cc1a3d None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:40.980721 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:41.000621 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28763fbe-173a-49bc-995c-8089bb4a730d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:41.001275 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28763fbe-173a-49bc-995c-8089bb4a730d None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:41.001562 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28763fbe-173a-49bc-995c-8089bb4a730d None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:41.001832 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28763fbe-173a-49bc-995c-8089bb4a730d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:41.020423 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28763fbe-173a-49bc-995c-8089bb4a730d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:00:41.031869 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-28763fbe-173a-49bc-995c-8089bb4a730d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 12:00:41.031869 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:00:41.031869 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:00:41.031869 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:00:41.031869 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:00:41.031869 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:00:41.031869 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:00:41.031869 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:00:41.031869 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:00:41.031869 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:00:41.031869 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:00:41.031869 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:00:41.031869 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:00:41.031869 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:00:41.031869 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:00:41.031869 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:00:41.032958 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:00:41.032958 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:00:41.032958 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:00:41.032958 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:00:41.032958 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:00:41.032958 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:00:41.032958 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:00:41.032958 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:00:41.032958 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:00:41.032958 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:00:41.032958 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:00:41.032958 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:00:41.032958 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:00:41.032958 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:00:41.032958 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:00:41.032958 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:00:41.033863 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:00:41.033863 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:00:41.033863 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:00:41.033863 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:00:41.033863 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:00:41.033863 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:00:41.033863 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:00:41.033863 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:00:41.033863 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:00:41.033863 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:00:41.033863 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 12:00:41.033863 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:00:41.033863 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 85/166] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:00:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:00:41.033863 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28763fbe-173a-49bc-995c-8089bb4a730d None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:41.083296 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 82/167] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:00:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 12:00:41.089256 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07eaf1-99af-49a9-8f12-2fbcb2741dfd None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:41.089852 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07eaf1-99af-49a9-8f12-2fbcb2741dfd None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:41.090327 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07eaf1-99af-49a9-8f12-2fbcb2741dfd None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:41.092911 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 83/168] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:00:41 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:00:41.093851 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea07eaf1-99af-49a9-8f12-2fbcb2741dfd None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:41.117628 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:41.209155 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea07eaf1-99af-49a9-8f12-2fbcb2741dfd None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:41.236037 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28763fbe-173a-49bc-995c-8089bb4a730d None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:41.246721 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa23908-6ecd-43f3-97b1-1de2c2da4c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:41.247618 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa23908-6ecd-43f3-97b1-1de2c2da4c86 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:41.248665 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa23908-6ecd-43f3-97b1-1de2c2da4c86 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:41.249145 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa23908-6ecd-43f3-97b1-1de2c2da4c86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:41.250077 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa23908-6ecd-43f3-97b1-1de2c2da4c86 None admin] RBAC: Adding query filter params (role.id=b2403690f5ad443aad059f5ee024f61c, scope.project.id=f66cc7270ac04520ad4a66086d154421, user.id=06c7a037a0fa485aa111a5e7e792882f) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:00:41.250674 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa23908-6ecd-43f3-97b1-1de2c2da4c86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:41.252411 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa23908-6ecd-43f3-97b1-1de2c2da4c86 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:41.259618 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 84/169] 10.222.0.61 () {56 vars in 1374 bytes} [Fri Aug 26 12:00:41 2022] GET /identity/v3/role_assignments?user.id=06c7a037a0fa485aa111a5e7e792882f&scope.project.id=f66cc7270ac04520ad4a66086d154421&role.id=b2403690f5ad443aad059f5ee024f61c => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:41.260726 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa23908-6ecd-43f3-97b1-1de2c2da4c86 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:41.337900 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d14f42-9936-4ca5-9b35-ae2ae11935c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:41.338505 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d14f42-9936-4ca5-9b35-ae2ae11935c5 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:41.338841 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d14f42-9936-4ca5-9b35-ae2ae11935c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:41.339117 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d14f42-9936-4ca5-9b35-ae2ae11935c5 None admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:41.339941 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d14f42-9936-4ca5-9b35-ae2ae11935c5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:00:41.340162 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d14f42-9936-4ca5-9b35-ae2ae11935c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:41.341620 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d14f42-9936-4ca5-9b35-ae2ae11935c5 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:41.350986 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 86/170] 10.222.0.61 () {56 vars in 1111 bytes} [Fri Aug 26 12:00:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:41.355983 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d14f42-9936-4ca5-9b35-ae2ae11935c5 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:41.363568 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:41.387319 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa23908-6ecd-43f3-97b1-1de2c2da4c86 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:41.427937 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:41.446697 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edc2dbe-0dd2-4631-8a25-d0e6e6769946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:41.447350 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc2dbe-0dd2-4631-8a25-d0e6e6769946 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:41.447635 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc2dbe-0dd2-4631-8a25-d0e6e6769946 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:41.447911 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc2dbe-0dd2-4631-8a25-d0e6e6769946 None admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:41.448724 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edc2dbe-0dd2-4631-8a25-d0e6e6769946 None admin] RBAC: Adding query filter params (name=member) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:00:41.449017 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edc2dbe-0dd2-4631-8a25-d0e6e6769946 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:41.450521 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edc2dbe-0dd2-4631-8a25-d0e6e6769946 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:41.460581 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 85/171] 10.222.0.61 () {56 vars in 1101 bytes} [Fri Aug 26 12:00:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:41.461917 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e6ce2-a559-4161-82a6-3f23b893c2a1 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:41.462207 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e6ce2-a559-4161-82a6-3f23b893c2a1 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:41.462480 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e6ce2-a559-4161-82a6-3f23b893c2a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:41.484097 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-021e6ce2-a559-4161-82a6-3f23b893c2a1 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:00:41.485450 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-021e6ce2-a559-4161-82a6-3f23b893c2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:00:41.492233 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d14f42-9936-4ca5-9b35-ae2ae11935c5 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:41.540474 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca6c42d-4377-4193-96fc-3d013c7d9a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:41.541464 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca6c42d-4377-4193-96fc-3d013c7d9a5d None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:41.541950 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca6c42d-4377-4193-96fc-3d013c7d9a5d None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:41.542404 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca6c42d-4377-4193-96fc-3d013c7d9a5d None admin] PATH_INFO: `/v3/projects/f66cc7270ac04520ad4a66086d154421` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:41.544325 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca6c42d-4377-4193-96fc-3d013c7d9a5d None admin] RBAC: Authorizing `identity:get_project(project_id=f66cc7270ac04520ad4a66086d154421)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:41.546415 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca6c42d-4377-4193-96fc-3d013c7d9a5d None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:41.551233 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 87/172] 10.222.0.61 () {56 vars in 1186 bytes} [Fri Aug 26 12:00:41 2022] GET /identity/v3/projects/f66cc7270ac04520ad4a66086d154421 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:41.551233 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c69d77-781a-42cc-8a04-fae0ff820dde None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:41.551233 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c69d77-781a-42cc-8a04-fae0ff820dde None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:41.551233 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c69d77-781a-42cc-8a04-fae0ff820dde None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:41.551562 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 88/173] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:00:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:00:41.552146 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c69d77-781a-42cc-8a04-fae0ff820dde None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:41.573038 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 86/174] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:00:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:00:41.575345 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cdee1d-03a9-4869-b514-f93b66f9ef6f None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:41.575630 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cdee1d-03a9-4869-b514-f93b66f9ef6f None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:41.575898 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cdee1d-03a9-4869-b514-f93b66f9ef6f None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:41.577723 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 87/175] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:00:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:00:41.582305 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cdee1d-03a9-4869-b514-f93b66f9ef6f None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:41.668854 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:41.684474 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c69d77-781a-42cc-8a04-fae0ff820dde None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:41.707577 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:41.723133 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cdee1d-03a9-4869-b514-f93b66f9ef6f None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:41.724080 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8569b0-69ee-4416-b64e-3e062c3d570a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:41.724773 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8569b0-69ee-4416-b64e-3e062c3d570a None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:41.725123 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8569b0-69ee-4416-b64e-3e062c3d570a None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:41.725462 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8569b0-69ee-4416-b64e-3e062c3d570a None admin] PATH_INFO: `/v3/users/06c7a037a0fa485aa111a5e7e792882f` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:41.726724 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8569b0-69ee-4416-b64e-3e062c3d570a None admin] RBAC: Authorizing `identity:get_user(user_id=06c7a037a0fa485aa111a5e7e792882f)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:41.728319 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8569b0-69ee-4416-b64e-3e062c3d570a None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:41.730188 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 89/176] 10.222.0.61 () {56 vars in 1177 bytes} [Fri Aug 26 12:00:41 2022] GET /identity/v3/users/06c7a037a0fa485aa111a5e7e792882f => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:41.736025 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d837d-154a-421d-81d3-a09070742d49 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:41.736440 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d837d-154a-421d-81d3-a09070742d49 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:41.736792 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d837d-154a-421d-81d3-a09070742d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:41.758403 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-c95d837d-154a-421d-81d3-a09070742d49 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:00:41.759697 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-c95d837d-154a-421d-81d3-a09070742d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:00:41.773339 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096ae59b-c4f1-438f-9429-2de7a3369e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:41.774424 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ae59b-c4f1-438f-9429-2de7a3369e35 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:41.775298 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ae59b-c4f1-438f-9429-2de7a3369e35 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:41.776029 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ae59b-c4f1-438f-9429-2de7a3369e35 None admin] PATH_INFO: `/v3/users/06c7a037a0fa485aa111a5e7e792882f` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:41.777548 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096ae59b-c4f1-438f-9429-2de7a3369e35 None admin] RBAC: Authorizing `identity:get_user(user_id=06c7a037a0fa485aa111a5e7e792882f)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:41.780074 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096ae59b-c4f1-438f-9429-2de7a3369e35 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:41.782513 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 88/177] 10.222.0.61 () {56 vars in 1177 bytes} [Fri Aug 26 12:00:41 2022] GET /identity/v3/users/06c7a037a0fa485aa111a5e7e792882f => generated 275 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:41.787364 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096ae59b-c4f1-438f-9429-2de7a3369e35 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:41.838173 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 90/178] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:00:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:00:41.839585 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b0188-73e5-46fa-b0fe-ae798d0008e0 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:41.839809 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b0188-73e5-46fa-b0fe-ae798d0008e0 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:41.840079 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b0188-73e5-46fa-b0fe-ae798d0008e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:41.859153 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-fa5b0188-73e5-46fa-b0fe-ae798d0008e0 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:00:41.860513 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-fa5b0188-73e5-46fa-b0fe-ae798d0008e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:00:41.920500 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096ae59b-c4f1-438f-9429-2de7a3369e35 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:41.933874 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 91/179] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:00:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:00:41.934805 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5b0188-73e5-46fa-b0fe-ae798d0008e0 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:41.972332 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7598a2e3-ef94-4112-8eb6-7fbebdfce38f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:41.973314 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7598a2e3-ef94-4112-8eb6-7fbebdfce38f None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:41.973808 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7598a2e3-ef94-4112-8eb6-7fbebdfce38f None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:41.974275 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7598a2e3-ef94-4112-8eb6-7fbebdfce38f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:41.975225 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7598a2e3-ef94-4112-8eb6-7fbebdfce38f None admin] RBAC: Adding query filter params (role.id=70ec1ba484714c8ca3adb8fe5845107c, scope.project.id=f66cc7270ac04520ad4a66086d154421, user.id=d101ea12e0cd40e0b25022bf649ac734) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:00:41.975730 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7598a2e3-ef94-4112-8eb6-7fbebdfce38f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:41.977770 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7598a2e3-ef94-4112-8eb6-7fbebdfce38f None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:41.983643 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:41.986324 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 89/180] 10.222.0.61 () {56 vars in 1374 bytes} [Fri Aug 26 12:00:41 2022] GET /identity/v3/role_assignments?user.id=d101ea12e0cd40e0b25022bf649ac734&scope.project.id=f66cc7270ac04520ad4a66086d154421&role.id=70ec1ba484714c8ca3adb8fe5845107c => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:41.987644 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7598a2e3-ef94-4112-8eb6-7fbebdfce38f None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:42.000244 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:42.109810 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5b0188-73e5-46fa-b0fe-ae798d0008e0 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:42.160668 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7598a2e3-ef94-4112-8eb6-7fbebdfce38f None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:42.172974 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac1c549-8527-45be-97b8-942669539af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:42.173841 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac1c549-8527-45be-97b8-942669539af1 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:42.174445 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac1c549-8527-45be-97b8-942669539af1 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:42.174893 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac1c549-8527-45be-97b8-942669539af1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:42.213100 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c232e8-3300-4022-ac79-be976230d3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:42.213722 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c232e8-3300-4022-ac79-be976230d3fb None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:42.214004 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c232e8-3300-4022-ac79-be976230d3fb None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:42.214277 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c232e8-3300-4022-ac79-be976230d3fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:42.262164 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac1c549-8527-45be-97b8-942669539af1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:42.264517 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac1c549-8527-45be-97b8-942669539af1 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:42.294121 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c232e8-3300-4022-ac79-be976230d3fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:42.296109 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c232e8-3300-4022-ac79-be976230d3fb None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:42.305412 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:42.320689 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:42.377358 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 92/181] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:00:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 12:00:42.378183 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac1c549-8527-45be-97b8-942669539af1 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:42.403122 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 90/182] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:00:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 12:00:42.404457 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b0bd7c-aef0-4c44-80e9-24adc2b2f33d None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:42.404836 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b0bd7c-aef0-4c44-80e9-24adc2b2f33d None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:42.405115 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b0bd7c-aef0-4c44-80e9-24adc2b2f33d None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:42.406362 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 91/183] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:00:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:00:42.407396 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b0bd7c-aef0-4c44-80e9-24adc2b2f33d None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:42.479068 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac1c549-8527-45be-97b8-942669539af1 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:42.509327 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b0bd7c-aef0-4c44-80e9-24adc2b2f33d None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:42.516575 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:42.518139 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d280d8-8c76-4a38-9c64-858fb119bd0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:42.518907 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d280d8-8c76-4a38-9c64-858fb119bd0d None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:42.519224 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d280d8-8c76-4a38-9c64-858fb119bd0d None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:42.519502 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d280d8-8c76-4a38-9c64-858fb119bd0d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:42.529290 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d280d8-8c76-4a38-9c64-858fb119bd0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:00:42.540696 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-d0d280d8-8c76-4a38-9c64-858fb119bd0d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 12:00:42.540696 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:00:42.540696 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:00:42.540696 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:00:42.540696 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:00:42.540696 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:00:42.540696 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:00:42.540696 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:00:42.540696 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:00:42.540696 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:00:42.540696 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:00:42.540696 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:00:42.540696 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:00:42.540696 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:00:42.540696 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:00:42.540696 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:00:42.541591 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:00:42.541591 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:00:42.541591 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:00:42.541591 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:00:42.541591 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:00:42.541591 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:00:42.541591 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:00:42.541591 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:00:42.541591 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:00:42.541591 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:00:42.541591 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:00:42.541591 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:00:42.541591 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:00:42.541591 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:00:42.541591 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:00:42.541591 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:00:42.542392 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:00:42.542392 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:00:42.542392 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:00:42.542392 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:00:42.542392 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:00:42.542392 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:00:42.542392 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:00:42.542392 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:00:42.542392 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:00:42.542392 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:00:42.542392 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 12:00:42.542392 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:00:42.542392 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 93/184] 10.222.0.61 () {56 vars in 1096 bytes} [Fri Aug 26 12:00:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:00:42.543056 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d280d8-8c76-4a38-9c64-858fb119bd0d None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:42.545393 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5f24d5-90e7-4754-aecc-7ab45e397a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:42.546192 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f24d5-90e7-4754-aecc-7ab45e397a12 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:42.546480 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f24d5-90e7-4754-aecc-7ab45e397a12 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:42.546751 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f24d5-90e7-4754-aecc-7ab45e397a12 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:42.547140 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:42.558348 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5f24d5-90e7-4754-aecc-7ab45e397a12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:00:42.563677 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-5d5f24d5-90e7-4754-aecc-7ab45e397a12 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 12:00:42.563677 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:00:42.563677 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:00:42.563677 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:00:42.563677 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:00:42.563677 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:00:42.563677 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:00:42.563677 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:00:42.563677 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:00:42.563677 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:00:42.563677 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:00:42.563677 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:00:42.563677 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:00:42.563677 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:00:42.563677 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:00:42.563677 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:00:42.564695 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:00:42.564695 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:00:42.564695 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:00:42.564695 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:00:42.564695 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:00:42.564695 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:00:42.564695 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:00:42.564695 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:00:42.564695 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:00:42.564695 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:00:42.564695 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:00:42.564695 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:00:42.564695 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:00:42.564695 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:00:42.564695 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:00:42.564695 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:00:42.565524 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:00:42.565524 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:00:42.565524 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:00:42.565524 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:00:42.565524 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:00:42.565524 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:00:42.565524 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:00:42.565524 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:00:42.565524 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:00:42.565524 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:00:42.565524 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 12:00:42.565524 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:00:42.565524 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 92/185] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:00:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:00:42.565524 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5f24d5-90e7-4754-aecc-7ab45e397a12 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:42.643532 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d280d8-8c76-4a38-9c64-858fb119bd0d None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:42.663644 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5f24d5-90e7-4754-aecc-7ab45e397a12 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:42.676488 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f259ec57-93df-4b41-abe7-c36363b11855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:42.677106 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f259ec57-93df-4b41-abe7-c36363b11855 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:42.677382 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f259ec57-93df-4b41-abe7-c36363b11855 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:42.677648 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f259ec57-93df-4b41-abe7-c36363b11855 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:42.687868 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f259ec57-93df-4b41-abe7-c36363b11855 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:00:42.692514 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-f259ec57-93df-4b41-abe7-c36363b11855 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 12:00:42.692514 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:00:42.692514 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:00:42.692514 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:00:42.692514 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:00:42.692514 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:00:42.692514 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:00:42.692514 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:00:42.692514 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:00:42.692514 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:00:42.692514 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:00:42.692514 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:00:42.692514 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:00:42.692514 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:00:42.692514 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:00:42.692514 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:00:42.693427 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:00:42.693427 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:00:42.693427 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:00:42.693427 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:00:42.693427 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:00:42.693427 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:00:42.693427 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:00:42.693427 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:00:42.693427 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:00:42.693427 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:00:42.693427 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:00:42.693427 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:00:42.693427 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:00:42.693427 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:00:42.693427 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:00:42.693427 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:00:42.694250 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:00:42.694250 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:00:42.694250 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:00:42.694250 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:00:42.694250 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:00:42.694250 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:00:42.694250 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:00:42.694250 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:00:42.694250 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:00:42.694250 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:00:42.694250 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 12:00:42.694250 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:00:42.694250 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 94/186] 10.222.0.61 () {56 vars in 1114 bytes} [Fri Aug 26 12:00:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:00:42.696247 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f259ec57-93df-4b41-abe7-c36363b11855 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:42.700743 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4073f4e6-ee3f-4afb-a809-18486d0eae01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:42.701518 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4073f4e6-ee3f-4afb-a809-18486d0eae01 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:42.701815 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4073f4e6-ee3f-4afb-a809-18486d0eae01 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:42.702087 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4073f4e6-ee3f-4afb-a809-18486d0eae01 None admin] PATH_INFO: `/v3/projects/f66cc7270ac04520ad4a66086d154421` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:42.703193 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4073f4e6-ee3f-4afb-a809-18486d0eae01 None admin] RBAC: Authorizing `identity:get_project(project_id=f66cc7270ac04520ad4a66086d154421)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:42.704671 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4073f4e6-ee3f-4afb-a809-18486d0eae01 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:42.706310 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 93/187] 10.222.0.61 () {56 vars in 1186 bytes} [Fri Aug 26 12:00:42 2022] GET /identity/v3/projects/f66cc7270ac04520ad4a66086d154421 => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:42.707421 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee2a60-3f2b-4a0e-b503-cf3f3313d914 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:42.707697 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee2a60-3f2b-4a0e-b503-cf3f3313d914 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:42.707963 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee2a60-3f2b-4a0e-b503-cf3f3313d914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:42.725295 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-2cee2a60-3f2b-4a0e-b503-cf3f3313d914 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:00:42.726473 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-2cee2a60-3f2b-4a0e-b503-cf3f3313d914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:00:42.754016 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:42.766408 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:42.805204 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 94/188] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:00:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:00:42.805204 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cee2a60-3f2b-4a0e-b503-cf3f3313d914 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:42.813437 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f259ec57-93df-4b41-abe7-c36363b11855 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:42.850518 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9dee67-88b8-4b86-8f61-3d032dd1577b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:42.851696 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9dee67-88b8-4b86-8f61-3d032dd1577b None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:42.851984 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9dee67-88b8-4b86-8f61-3d032dd1577b None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:42.852259 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9dee67-88b8-4b86-8f61-3d032dd1577b None admin] PATH_INFO: `/v3/projects/c2765307b2fd4cf08246c1e63147f07e` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:42.853905 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9dee67-88b8-4b86-8f61-3d032dd1577b None admin] RBAC: Authorizing `identity:get_project(project_id=c2765307b2fd4cf08246c1e63147f07e)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:42.856130 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9dee67-88b8-4b86-8f61-3d032dd1577b None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:42.859046 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 95/189] 10.222.0.61 () {56 vars in 1186 bytes} [Fri Aug 26 12:00:42 2022] GET /identity/v3/projects/c2765307b2fd4cf08246c1e63147f07e => generated 312 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:42.860060 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9dee67-88b8-4b86-8f61-3d032dd1577b None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:42.905323 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cee2a60-3f2b-4a0e-b503-cf3f3313d914 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:42.940085 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a8cc77-7c0a-4096-9375-b95758f84fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:42.940689 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a8cc77-7c0a-4096-9375-b95758f84fcb None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:42.940992 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a8cc77-7c0a-4096-9375-b95758f84fcb None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:42.941264 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a8cc77-7c0a-4096-9375-b95758f84fcb None admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:42.941924 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a8cc77-7c0a-4096-9375-b95758f84fcb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:00:42.942219 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a8cc77-7c0a-4096-9375-b95758f84fcb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:42.943639 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a8cc77-7c0a-4096-9375-b95758f84fcb None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:42.951824 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 95/190] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:00:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:42.952633 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a8cc77-7c0a-4096-9375-b95758f84fcb None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:42.980622 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:42.982117 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9dee67-88b8-4b86-8f61-3d032dd1577b None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:43.015632 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:43.021245 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de0fca2-66c0-427c-8415-ae5b054654cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:43.021849 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de0fca2-66c0-427c-8415-ae5b054654cc None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:43.022134 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de0fca2-66c0-427c-8415-ae5b054654cc None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:43.022387 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de0fca2-66c0-427c-8415-ae5b054654cc None admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:43.023045 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de0fca2-66c0-427c-8415-ae5b054654cc None admin] RBAC: Adding query filter params (name=reader) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:00:43.023373 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de0fca2-66c0-427c-8415-ae5b054654cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:43.024748 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de0fca2-66c0-427c-8415-ae5b054654cc None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:43.033166 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 96/191] 10.222.0.61 () {56 vars in 1101 bytes} [Fri Aug 26 12:00:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:43.033954 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de0fca2-66c0-427c-8415-ae5b054654cc None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:43.062561 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a8cc77-7c0a-4096-9375-b95758f84fcb None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:43.096698 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e88650-122b-4bed-b6b7-ca94b0773629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:43.097302 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e88650-122b-4bed-b6b7-ca94b0773629 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:43.097584 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e88650-122b-4bed-b6b7-ca94b0773629 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:43.097859 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e88650-122b-4bed-b6b7-ca94b0773629 None admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:43.098520 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e88650-122b-4bed-b6b7-ca94b0773629 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:00:43.098822 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e88650-122b-4bed-b6b7-ca94b0773629 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:43.100342 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e88650-122b-4bed-b6b7-ca94b0773629 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:43.109001 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 96/192] 10.222.0.61 () {56 vars in 1111 bytes} [Fri Aug 26 12:00:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:43.109588 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e88650-122b-4bed-b6b7-ca94b0773629 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:43.138200 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de0fca2-66c0-427c-8415-ae5b054654cc None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:43.170984 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0b58ca-03b8-4495-8a7b-5e7b19441ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:43.171549 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0b58ca-03b8-4495-8a7b-5e7b19441ee5 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:43.171828 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0b58ca-03b8-4495-8a7b-5e7b19441ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:43.172109 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0b58ca-03b8-4495-8a7b-5e7b19441ee5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:43.172834 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0b58ca-03b8-4495-8a7b-5e7b19441ee5 None admin] RBAC: Adding query filter params (role.id=a60f381814fe4c1da324d13c8e7d5d20, scope.project.id=f66cc7270ac04520ad4a66086d154421, user.id=06c7a037a0fa485aa111a5e7e792882f) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:00:43.173140 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0b58ca-03b8-4495-8a7b-5e7b19441ee5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:43.174585 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0b58ca-03b8-4495-8a7b-5e7b19441ee5 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:43.180312 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 97/193] 10.222.0.61 () {56 vars in 1374 bytes} [Fri Aug 26 12:00:43 2022] GET /identity/v3/role_assignments?user.id=06c7a037a0fa485aa111a5e7e792882f&scope.project.id=f66cc7270ac04520ad4a66086d154421&role.id=a60f381814fe4c1da324d13c8e7d5d20 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:43.181093 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0b58ca-03b8-4495-8a7b-5e7b19441ee5 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:43.213344 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:43.228792 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e88650-122b-4bed-b6b7-ca94b0773629 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:43.256815 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:43.269856 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61624acc-2765-4bbd-b8ff-b5c6751361c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:43.270573 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61624acc-2765-4bbd-b8ff-b5c6751361c4 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:43.270771 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61624acc-2765-4bbd-b8ff-b5c6751361c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:43.271050 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61624acc-2765-4bbd-b8ff-b5c6751361c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:43.271852 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61624acc-2765-4bbd-b8ff-b5c6751361c4 None admin] RBAC: Adding query filter params (role.id=b2403690f5ad443aad059f5ee024f61c, scope.project.id=c2765307b2fd4cf08246c1e63147f07e, user.id=06c7a037a0fa485aa111a5e7e792882f) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:00:43.272167 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61624acc-2765-4bbd-b8ff-b5c6751361c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:43.273658 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61624acc-2765-4bbd-b8ff-b5c6751361c4 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:43.280710 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 97/194] 10.222.0.61 () {56 vars in 1374 bytes} [Fri Aug 26 12:00:43 2022] GET /identity/v3/role_assignments?user.id=06c7a037a0fa485aa111a5e7e792882f&scope.project.id=c2765307b2fd4cf08246c1e63147f07e&role.id=b2403690f5ad443aad059f5ee024f61c => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:43.281579 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61624acc-2765-4bbd-b8ff-b5c6751361c4 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:43.328524 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0b58ca-03b8-4495-8a7b-5e7b19441ee5 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:43.371065 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e118e0fc-0130-4975-a30f-2288bf71a0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:43.371748 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e118e0fc-0130-4975-a30f-2288bf71a0ce None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:43.372028 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e118e0fc-0130-4975-a30f-2288bf71a0ce None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:43.372301 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e118e0fc-0130-4975-a30f-2288bf71a0ce None admin] PATH_INFO: `/v3/users/8f9b8686397a47409e5d4b2d855191a2` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:43.394100 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e118e0fc-0130-4975-a30f-2288bf71a0ce None admin] RBAC: Authorizing `identity:get_user(user_id=8f9b8686397a47409e5d4b2d855191a2)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:43.396253 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e118e0fc-0130-4975-a30f-2288bf71a0ce None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:43.398591 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 98/195] 10.222.0.61 () {56 vars in 1177 bytes} [Fri Aug 26 12:00:43 2022] GET /identity/v3/users/8f9b8686397a47409e5d4b2d855191a2 => generated 283 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:43.399741 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e118e0fc-0130-4975-a30f-2288bf71a0ce None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:43.416404 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61624acc-2765-4bbd-b8ff-b5c6751361c4 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:43.461174 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082e2bb6-9a0f-4596-8a68-e10a060e9f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:43.462119 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082e2bb6-9a0f-4596-8a68-e10a060e9f0a None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:43.463163 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082e2bb6-9a0f-4596-8a68-e10a060e9f0a None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:43.463163 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082e2bb6-9a0f-4596-8a68-e10a060e9f0a None admin] PATH_INFO: `/v3/users/61f5e5a725cc424eb12ca035558089e3` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:43.464454 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082e2bb6-9a0f-4596-8a68-e10a060e9f0a None admin] RBAC: Authorizing `identity:get_user(user_id=61f5e5a725cc424eb12ca035558089e3)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:43.466073 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082e2bb6-9a0f-4596-8a68-e10a060e9f0a None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:43.468836 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 98/196] 10.222.0.61 () {56 vars in 1177 bytes} [Fri Aug 26 12:00:43 2022] GET /identity/v3/users/61f5e5a725cc424eb12ca035558089e3 => generated 289 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:43.469620 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082e2bb6-9a0f-4596-8a68-e10a060e9f0a None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:43.493637 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:43.533684 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e118e0fc-0130-4975-a30f-2288bf71a0ce None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:43.550327 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:43.568827 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed98c2dc-860b-422d-aa91-e7b558cb036a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:43.569455 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed98c2dc-860b-422d-aa91-e7b558cb036a None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:43.569733 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed98c2dc-860b-422d-aa91-e7b558cb036a None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:43.570013 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed98c2dc-860b-422d-aa91-e7b558cb036a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:43.571137 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed98c2dc-860b-422d-aa91-e7b558cb036a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:43.572622 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed98c2dc-860b-422d-aa91-e7b558cb036a None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:43.574192 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 99/197] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:00:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:43.575487 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed98c2dc-860b-422d-aa91-e7b558cb036a None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:43.580010 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082e2bb6-9a0f-4596-8a68-e10a060e9f0a None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:43.612952 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226b796c-b24f-46c4-a1df-5d682cc3f99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:43.613573 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b796c-b24f-46c4-a1df-5d682cc3f99c None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:43.613853 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b796c-b24f-46c4-a1df-5d682cc3f99c None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:43.614128 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b796c-b24f-46c4-a1df-5d682cc3f99c None admin] PATH_INFO: `/v3/users/8f9b8686397a47409e5d4b2d855191a2` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:43.615287 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226b796c-b24f-46c4-a1df-5d682cc3f99c None admin] RBAC: Authorizing `identity:get_user(user_id=8f9b8686397a47409e5d4b2d855191a2)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:43.616741 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226b796c-b24f-46c4-a1df-5d682cc3f99c None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:43.618318 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 99/198] 10.222.0.61 () {56 vars in 1177 bytes} [Fri Aug 26 12:00:43 2022] GET /identity/v3/users/8f9b8686397a47409e5d4b2d855191a2 => generated 283 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:43.618318 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226b796c-b24f-46c4-a1df-5d682cc3f99c None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:43.674194 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed98c2dc-860b-422d-aa91-e7b558cb036a None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:43.707779 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0447e1a-db4a-429c-8f67-29aa71e6326e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:43.708370 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0447e1a-db4a-429c-8f67-29aa71e6326e None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:43.708652 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0447e1a-db4a-429c-8f67-29aa71e6326e None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:43.708926 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0447e1a-db4a-429c-8f67-29aa71e6326e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:43.710820 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:43.722750 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226b796c-b24f-46c4-a1df-5d682cc3f99c None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:43.758493 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bae67f-8323-4480-93db-bd54ac82cf5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:43.759093 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bae67f-8323-4480-93db-bd54ac82cf5b None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:43.759398 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bae67f-8323-4480-93db-bd54ac82cf5b None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:43.759668 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bae67f-8323-4480-93db-bd54ac82cf5b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:43.764763 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:43.780435 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0447e1a-db4a-429c-8f67-29aa71e6326e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:43.781909 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0447e1a-db4a-429c-8f67-29aa71e6326e None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:43.825679 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bae67f-8323-4480-93db-bd54ac82cf5b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:43.827828 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bae67f-8323-4480-93db-bd54ac82cf5b None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:43.879120 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 100/199] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:00:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 12:00:43.879449 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0447e1a-db4a-429c-8f67-29aa71e6326e None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:43.912819 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:43.926754 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 100/200] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:00:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 12:00:43.927606 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bae67f-8323-4480-93db-bd54ac82cf5b None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:43.962071 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:43.979240 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0447e1a-db4a-429c-8f67-29aa71e6326e None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:44.014121 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4827f41-f219-4100-b183-078f31e0c5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:44.014744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4827f41-f219-4100-b183-078f31e0c5f9 None admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:44.015021 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4827f41-f219-4100-b183-078f31e0c5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:44.015321 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4827f41-f219-4100-b183-078f31e0c5f9 None admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:44.015961 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4827f41-f219-4100-b183-078f31e0c5f9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:44.017398 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4827f41-f219-4100-b183-078f31e0c5f9 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:44.026042 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bae67f-8323-4480-93db-bd54ac82cf5b None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:44.047238 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 101/201] 10.222.0.61 () {60 vars in 1132 bytes} [Fri Aug 26 12:00:43 2022] POST /identity/v3/users => generated 297 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:00:44.047238 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4827f41-f219-4100-b183-078f31e0c5f9 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:44.064432 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d16aa82-9231-4da4-abbb-543dc8393ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:44.065064 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d16aa82-9231-4da4-abbb-543dc8393ff5 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:44.065343 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d16aa82-9231-4da4-abbb-543dc8393ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:44.065963 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d16aa82-9231-4da4-abbb-543dc8393ff5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:44.133744 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:44.148689 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d16aa82-9231-4da4-abbb-543dc8393ff5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:44.150188 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d16aa82-9231-4da4-abbb-543dc8393ff5 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:44.175076 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4827f41-f219-4100-b183-078f31e0c5f9 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:44.213272 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ba132f-4a26-4709-84cb-bd668cef1813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:44.213914 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ba132f-4a26-4709-84cb-bd668cef1813 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:44.214193 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ba132f-4a26-4709-84cb-bd668cef1813 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:44.214472 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ba132f-4a26-4709-84cb-bd668cef1813 None admin] PATH_INFO: `/v3/projects/16904d4c978c4c12abd64d625faa3eda` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:44.215646 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ba132f-4a26-4709-84cb-bd668cef1813 None admin] RBAC: Authorizing `identity:get_project(project_id=16904d4c978c4c12abd64d625faa3eda)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:44.217757 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ba132f-4a26-4709-84cb-bd668cef1813 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:44.219539 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 102/202] 10.222.0.61 () {56 vars in 1186 bytes} [Fri Aug 26 12:00:44 2022] GET /identity/v3/projects/16904d4c978c4c12abd64d625faa3eda => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:44.220306 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ba132f-4a26-4709-84cb-bd668cef1813 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:44.235737 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:44.278805 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 101/203] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:00:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 12:00:44.279730 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d16aa82-9231-4da4-abbb-543dc8393ff5 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:44.360260 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ba132f-4a26-4709-84cb-bd668cef1813 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:44.392340 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:44.406883 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf53b3b-0b03-4da0-bee5-d2c35d0d3c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:44.406883 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf53b3b-0b03-4da0-bee5-d2c35d0d3c81 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:44.407156 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf53b3b-0b03-4da0-bee5-d2c35d0d3c81 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:44.407388 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf53b3b-0b03-4da0-bee5-d2c35d0d3c81 None admin] PATH_INFO: `/v3/projects/f66cc7270ac04520ad4a66086d154421` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:44.408535 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf53b3b-0b03-4da0-bee5-d2c35d0d3c81 None admin] RBAC: Authorizing `identity:get_project(project_id=f66cc7270ac04520ad4a66086d154421)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:44.409990 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf53b3b-0b03-4da0-bee5-d2c35d0d3c81 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:44.411119 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d16aa82-9231-4da4-abbb-543dc8393ff5 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:44.411730 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 103/204] 10.222.0.61 () {56 vars in 1186 bytes} [Fri Aug 26 12:00:44 2022] GET /identity/v3/projects/f66cc7270ac04520ad4a66086d154421 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:44.412488 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf53b3b-0b03-4da0-bee5-d2c35d0d3c81 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:44.450692 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d58a7-78f9-44e8-80b6-2d80a02b0487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:44.452191 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d58a7-78f9-44e8-80b6-2d80a02b0487 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:44.452653 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d58a7-78f9-44e8-80b6-2d80a02b0487 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:44.453104 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d58a7-78f9-44e8-80b6-2d80a02b0487 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:44.454036 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139d58a7-78f9-44e8-80b6-2d80a02b0487 None admin] RBAC: Adding query filter params (role.id=70ec1ba484714c8ca3adb8fe5845107c, scope.project.id=16904d4c978c4c12abd64d625faa3eda, user.id=8f9b8686397a47409e5d4b2d855191a2) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:00:44.454521 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139d58a7-78f9-44e8-80b6-2d80a02b0487 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:44.456757 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139d58a7-78f9-44e8-80b6-2d80a02b0487 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:44.465075 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 102/205] 10.222.0.61 () {56 vars in 1374 bytes} [Fri Aug 26 12:00:44 2022] GET /identity/v3/role_assignments?user.id=8f9b8686397a47409e5d4b2d855191a2&scope.project.id=16904d4c978c4c12abd64d625faa3eda&role.id=70ec1ba484714c8ca3adb8fe5845107c => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:44.466128 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d58a7-78f9-44e8-80b6-2d80a02b0487 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:44.526939 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:44.574508 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf53b3b-0b03-4da0-bee5-d2c35d0d3c81 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:44.636847 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74db0568-1b5a-4bc4-b3bb-51e1b1b261f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:44.637803 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74db0568-1b5a-4bc4-b3bb-51e1b1b261f6 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:44.638263 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74db0568-1b5a-4bc4-b3bb-51e1b1b261f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:44.639439 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74db0568-1b5a-4bc4-b3bb-51e1b1b261f6 None admin] PATH_INFO: `/v3/projects/16904d4c978c4c12abd64d625faa3eda` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:44.640876 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74db0568-1b5a-4bc4-b3bb-51e1b1b261f6 None admin] RBAC: Authorizing `identity:get_project(project_id=16904d4c978c4c12abd64d625faa3eda)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:44.642646 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74db0568-1b5a-4bc4-b3bb-51e1b1b261f6 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:44.644861 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 104/206] 10.222.0.61 () {56 vars in 1186 bytes} [Fri Aug 26 12:00:44 2022] GET /identity/v3/projects/16904d4c978c4c12abd64d625faa3eda => generated 302 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:44.650377 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74db0568-1b5a-4bc4-b3bb-51e1b1b261f6 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:44.671276 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d58a7-78f9-44e8-80b6-2d80a02b0487 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:44.721166 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e28b835-71f2-49fc-b128-64b10905af29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:44.721792 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e28b835-71f2-49fc-b128-64b10905af29 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:44.722065 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e28b835-71f2-49fc-b128-64b10905af29 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:44.722335 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e28b835-71f2-49fc-b128-64b10905af29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:44.723067 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e28b835-71f2-49fc-b128-64b10905af29 None admin] RBAC: Adding query filter params (role.id=12a2a8379bd241a098646f1c8d881614, scope.project.id=f66cc7270ac04520ad4a66086d154421, user.id=61f5e5a725cc424eb12ca035558089e3) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:00:44.723394 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e28b835-71f2-49fc-b128-64b10905af29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:44.724845 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e28b835-71f2-49fc-b128-64b10905af29 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:44.735628 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 103/207] 10.222.0.61 () {56 vars in 1374 bytes} [Fri Aug 26 12:00:44 2022] GET /identity/v3/role_assignments?user.id=61f5e5a725cc424eb12ca035558089e3&scope.project.id=f66cc7270ac04520ad4a66086d154421&role.id=12a2a8379bd241a098646f1c8d881614 => generated 608 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:44.791818 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74db0568-1b5a-4bc4-b3bb-51e1b1b261f6 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:44.833075 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16191dd7-d80e-4156-9a23-4a217a8f00c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:44.834073 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16191dd7-d80e-4156-9a23-4a217a8f00c9 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:44.834561 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16191dd7-d80e-4156-9a23-4a217a8f00c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:44.835376 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16191dd7-d80e-4156-9a23-4a217a8f00c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:44.836425 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16191dd7-d80e-4156-9a23-4a217a8f00c9 None admin] RBAC: Adding query filter params (role.id=a60f381814fe4c1da324d13c8e7d5d20, scope.project.id=16904d4c978c4c12abd64d625faa3eda, user.id=8f9b8686397a47409e5d4b2d855191a2) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:00:44.836963 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16191dd7-d80e-4156-9a23-4a217a8f00c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:44.838674 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16191dd7-d80e-4156-9a23-4a217a8f00c9 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:44.848248 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 105/208] 10.222.0.61 () {56 vars in 1374 bytes} [Fri Aug 26 12:00:44 2022] GET /identity/v3/role_assignments?user.id=8f9b8686397a47409e5d4b2d855191a2&scope.project.id=16904d4c978c4c12abd64d625faa3eda&role.id=a60f381814fe4c1da324d13c8e7d5d20 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:46.183437 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a666cb-b682-4302-83c0-f2fde51fd6f9 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:46.183951 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a666cb-b682-4302-83c0-f2fde51fd6f9 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:46.184025 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a666cb-b682-4302-83c0-f2fde51fd6f9 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:46.186117 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 104/209] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:00:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:00:46.191702 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16af855-d950-4628-8262-c280ff929305 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:46.192128 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16af855-d950-4628-8262-c280ff929305 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:46.192325 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16af855-d950-4628-8262-c280ff929305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:46.210934 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-b16af855-d950-4628-8262-c280ff929305 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:00:46.212633 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-b16af855-d950-4628-8262-c280ff929305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:00:46.214852 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:46.286294 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 106/210] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:00:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 12:00:46.344595 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c85fc-3aeb-4ef7-83f8-b1e2132e3c23 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:46.345466 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c85fc-3aeb-4ef7-83f8-b1e2132e3c23 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:46.346089 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c85fc-3aeb-4ef7-83f8-b1e2132e3c23 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:46.348100 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 105/211] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:00:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:00:46.353763 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aa8e24-1cbf-4f6e-a86c-d6291edb5832 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:46.354307 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aa8e24-1cbf-4f6e-a86c-d6291edb5832 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:46.354762 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aa8e24-1cbf-4f6e-a86c-d6291edb5832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:46.376574 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-04aa8e24-1cbf-4f6e-a86c-d6291edb5832 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:00:46.378070 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-04aa8e24-1cbf-4f6e-a86c-d6291edb5832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:00:46.455277 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 107/212] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:00:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:00:46.548752 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd78e48b-7aa0-4aaf-9d13-288859a48339 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:46.549405 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd78e48b-7aa0-4aaf-9d13-288859a48339 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:46.549889 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd78e48b-7aa0-4aaf-9d13-288859a48339 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:46.551476 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 106/213] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:00:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:00:46.556976 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8824483b-7509-4e88-8278-3b48a7f09786 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:46.557503 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8824483b-7509-4e88-8278-3b48a7f09786 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:46.558001 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8824483b-7509-4e88-8278-3b48a7f09786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:46.576268 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-8824483b-7509-4e88-8278-3b48a7f09786 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:00:46.577804 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-8824483b-7509-4e88-8278-3b48a7f09786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:00:46.581391 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd78e48b-7aa0-4aaf-9d13-288859a48339 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:46.601209 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:46.661355 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:46.668506 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 108/214] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:00:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:00:46.702997 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd78e48b-7aa0-4aaf-9d13-288859a48339 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:46.737209 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8824483b-7509-4e88-8278-3b48a7f09786 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:46.750725 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b70e4c8-7ab4-4c17-b765-7b99394fda8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:46.751514 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b70e4c8-7ab4-4c17-b765-7b99394fda8b None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:46.751798 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b70e4c8-7ab4-4c17-b765-7b99394fda8b None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:46.752073 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b70e4c8-7ab4-4c17-b765-7b99394fda8b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:46.764067 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b70e4c8-7ab4-4c17-b765-7b99394fda8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:00:46.771268 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-5b70e4c8-7ab4-4c17-b765-7b99394fda8b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 12:00:46.771268 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:00:46.771268 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:00:46.771268 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:00:46.771268 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:00:46.771268 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:00:46.771268 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:00:46.771268 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:00:46.771268 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:00:46.771268 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:00:46.771268 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:00:46.771268 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:00:46.771268 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:00:46.771268 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:00:46.771268 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:00:46.771268 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:00:46.772152 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:00:46.772152 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:00:46.772152 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:00:46.772152 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:00:46.772152 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:00:46.772152 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:00:46.772152 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:00:46.772152 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:00:46.772152 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:00:46.772152 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:00:46.772152 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:00:46.772152 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:00:46.772152 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:00:46.772152 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:00:46.772152 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:00:46.772152 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:00:46.772941 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:00:46.772941 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:00:46.772941 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:00:46.772941 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:00:46.772941 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:00:46.772941 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:00:46.772941 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:00:46.772941 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:00:46.772941 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:00:46.772941 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:00:46.772941 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 12:00:46.772941 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:00:46.772941 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 107/215] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:00:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 12:00:46.781029 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b70e4c8-7ab4-4c17-b765-7b99394fda8b None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:46.872570 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8824483b-7509-4e88-8278-3b48a7f09786 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:46.883537 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:46.896569 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b70e4c8-7ab4-4c17-b765-7b99394fda8b None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:46.912439 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3c5bca-2c7a-434b-b395-9d70643bc3d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:46.913209 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3c5bca-2c7a-434b-b395-9d70643bc3d8 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:46.913500 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3c5bca-2c7a-434b-b395-9d70643bc3d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:46.913777 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3c5bca-2c7a-434b-b395-9d70643bc3d8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:46.914890 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3c5bca-2c7a-434b-b395-9d70643bc3d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:46.916473 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3c5bca-2c7a-434b-b395-9d70643bc3d8 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:46.918308 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 109/216] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:00:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:00:46.919481 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d4675-2d9d-48ee-8605-639110e542ba None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:46.919970 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d4675-2d9d-48ee-8605-639110e542ba None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:46.920636 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63d4675-2d9d-48ee-8605-639110e542ba None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:46.922008 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 110/217] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:00:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:00:46.923414 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63d4675-2d9d-48ee-8605-639110e542ba None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:46.939771 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a93a8a-4553-4f8e-ab09-1e78d41585f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:46.940378 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a93a8a-4553-4f8e-ab09-1e78d41585f0 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:46.940678 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a93a8a-4553-4f8e-ab09-1e78d41585f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:46.940927 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a93a8a-4553-4f8e-ab09-1e78d41585f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:46.941572 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a93a8a-4553-4f8e-ab09-1e78d41585f0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:00:46.941869 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a93a8a-4553-4f8e-ab09-1e78d41585f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:46.943249 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a93a8a-4553-4f8e-ab09-1e78d41585f0 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:46.952604 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 108/218] 10.222.0.61 () {56 vars in 1101 bytes} [Fri Aug 26 12:00:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:46.952604 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a114ae05-4d71-4521-a4d4-f918a5b26123 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:46.952604 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a114ae05-4d71-4521-a4d4-f918a5b26123 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:46.952604 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a114ae05-4d71-4521-a4d4-f918a5b26123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:46.972068 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-a114ae05-4d71-4521-a4d4-f918a5b26123 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:00:46.973237 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-a114ae05-4d71-4521-a4d4-f918a5b26123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:00:46.997908 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:47.035088 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63d4675-2d9d-48ee-8605-639110e542ba None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:47.039369 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 109/219] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:00:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:00:47.043346 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a114ae05-4d71-4521-a4d4-f918a5b26123 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:47.071304 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822e5e9c-46e2-40f7-8123-8b79858964a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:47.071914 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822e5e9c-46e2-40f7-8123-8b79858964a0 None admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:47.072202 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822e5e9c-46e2-40f7-8123-8b79858964a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:47.072478 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822e5e9c-46e2-40f7-8123-8b79858964a0 None admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:47.073127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822e5e9c-46e2-40f7-8123-8b79858964a0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:47.074610 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822e5e9c-46e2-40f7-8123-8b79858964a0 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:47.098832 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 111/220] 10.222.0.61 () {60 vars in 1132 bytes} [Fri Aug 26 12:00:46 2022] POST /identity/v3/users => generated 297 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:00:47.110256 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822e5e9c-46e2-40f7-8123-8b79858964a0 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:47.122940 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:47.175905 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a114ae05-4d71-4521-a4d4-f918a5b26123 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:47.232028 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6da88f-183c-4b3c-94af-425a8ef8ef62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:47.233338 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6da88f-183c-4b3c-94af-425a8ef8ef62 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:47.233828 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6da88f-183c-4b3c-94af-425a8ef8ef62 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:47.234126 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6da88f-183c-4b3c-94af-425a8ef8ef62 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:47.251957 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6da88f-183c-4b3c-94af-425a8ef8ef62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:00:47.257097 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-4c6da88f-183c-4b3c-94af-425a8ef8ef62 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 12:00:47.257097 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:00:47.257097 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:00:47.257097 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:00:47.257097 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:00:47.257097 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:00:47.257097 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:00:47.257097 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:00:47.257097 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:00:47.257097 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:00:47.257097 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:00:47.257097 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:00:47.257097 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:00:47.257097 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:00:47.257097 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:00:47.257097 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:00:47.258234 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:00:47.258234 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:00:47.258234 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:00:47.258234 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:00:47.258234 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:00:47.258234 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:00:47.258234 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:00:47.258234 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:00:47.258234 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:00:47.258234 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:00:47.258234 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:00:47.258234 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:00:47.258234 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:00:47.258234 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:00:47.258234 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:00:47.258234 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:00:47.259051 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:00:47.259051 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:00:47.259051 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:00:47.259051 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:00:47.259051 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:00:47.259051 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:00:47.259051 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:00:47.259051 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:00:47.259051 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:00:47.259051 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:00:47.259051 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 12:00:47.259051 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:00:47.259051 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 110/221] 10.222.0.61 () {56 vars in 1096 bytes} [Fri Aug 26 12:00:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:00:47.265935 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6da88f-183c-4b3c-94af-425a8ef8ef62 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:47.266066 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822e5e9c-46e2-40f7-8123-8b79858964a0 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:47.300930 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:47.320131 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded8ecc-945a-427f-bb5a-5835e660f31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:47.320782 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded8ecc-945a-427f-bb5a-5835e660f31b None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:47.321057 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded8ecc-945a-427f-bb5a-5835e660f31b None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:47.321329 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded8ecc-945a-427f-bb5a-5835e660f31b None admin] PATH_INFO: `/v3/users/cb0932d41804458783f0b2063f06bd69` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:47.338242 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bded8ecc-945a-427f-bb5a-5835e660f31b None admin] RBAC: Authorizing `identity:get_user(user_id=cb0932d41804458783f0b2063f06bd69)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:47.339900 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bded8ecc-945a-427f-bb5a-5835e660f31b None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:47.342405 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 112/222] 10.222.0.61 () {56 vars in 1177 bytes} [Fri Aug 26 12:00:47 2022] GET /identity/v3/users/cb0932d41804458783f0b2063f06bd69 => generated 297 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:47.343715 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded8ecc-945a-427f-bb5a-5835e660f31b None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:47.402824 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6da88f-183c-4b3c-94af-425a8ef8ef62 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:47.443142 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:47.453645 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c282e846-d2d3-41f4-bf4d-ceb76a52e1b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:47.454261 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c282e846-d2d3-41f4-bf4d-ceb76a52e1b4 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:47.454815 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c282e846-d2d3-41f4-bf4d-ceb76a52e1b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:47.454920 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c282e846-d2d3-41f4-bf4d-ceb76a52e1b4 None admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:47.456116 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c282e846-d2d3-41f4-bf4d-ceb76a52e1b4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:00:47.456116 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c282e846-d2d3-41f4-bf4d-ceb76a52e1b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:47.457875 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c282e846-d2d3-41f4-bf4d-ceb76a52e1b4 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:47.468408 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 111/223] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:00:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:47.469510 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c282e846-d2d3-41f4-bf4d-ceb76a52e1b4 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:47.472157 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded8ecc-945a-427f-bb5a-5835e660f31b None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:47.515087 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b670bce6-a7f7-4bd9-b452-62c1673ec75b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:47.516073 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b670bce6-a7f7-4bd9-b452-62c1673ec75b None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:47.516695 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b670bce6-a7f7-4bd9-b452-62c1673ec75b None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:47.517237 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b670bce6-a7f7-4bd9-b452-62c1673ec75b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:47.528368 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b670bce6-a7f7-4bd9-b452-62c1673ec75b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:00:47.534604 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-b670bce6-a7f7-4bd9-b452-62c1673ec75b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 12:00:47.534604 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:00:47.534604 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:00:47.534604 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:00:47.534604 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:00:47.534604 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:00:47.534604 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:00:47.534604 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:00:47.534604 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:00:47.534604 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:00:47.534604 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:00:47.534604 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:00:47.534604 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:00:47.534604 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:00:47.534604 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:00:47.534604 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:00:47.535508 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:00:47.535508 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:00:47.535508 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:00:47.535508 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:00:47.535508 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:00:47.535508 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:00:47.535508 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:00:47.535508 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:00:47.535508 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:00:47.535508 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:00:47.535508 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:00:47.535508 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:00:47.535508 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:00:47.535508 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:00:47.535508 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:00:47.535508 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:00:47.536321 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:00:47.536321 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:00:47.536321 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:00:47.536321 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:00:47.536321 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:00:47.536321 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:00:47.536321 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:00:47.536321 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:00:47.536321 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:00:47.536321 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:00:47.536321 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 12:00:47.536321 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:00:47.538701 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 113/224] 10.222.0.61 () {56 vars in 1114 bytes} [Fri Aug 26 12:00:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:00:47.539806 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b670bce6-a7f7-4bd9-b452-62c1673ec75b None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:47.595542 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:47.602034 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c282e846-d2d3-41f4-bf4d-ceb76a52e1b4 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:47.647832 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6a40dd-dcf1-4efa-814b-7a9251bf2ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:47.649020 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6a40dd-dcf1-4efa-814b-7a9251bf2ef9 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:47.649476 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6a40dd-dcf1-4efa-814b-7a9251bf2ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:47.650047 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6a40dd-dcf1-4efa-814b-7a9251bf2ef9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:47.660108 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b670bce6-a7f7-4bd9-b452-62c1673ec75b None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:47.697429 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decd941f-ec91-4d22-b038-e9fbe500a996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:47.698178 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decd941f-ec91-4d22-b038-e9fbe500a996 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:47.698548 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decd941f-ec91-4d22-b038-e9fbe500a996 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:47.698825 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decd941f-ec91-4d22-b038-e9fbe500a996 None admin] PATH_INFO: `/v3/users/8f9b8686397a47409e5d4b2d855191a2` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:47.700461 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decd941f-ec91-4d22-b038-e9fbe500a996 None admin] RBAC: Authorizing `identity:get_user(user_id=8f9b8686397a47409e5d4b2d855191a2)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:47.701842 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decd941f-ec91-4d22-b038-e9fbe500a996 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:47.703762 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 114/225] 10.222.0.61 () {56 vars in 1177 bytes} [Fri Aug 26 12:00:47 2022] GET /identity/v3/users/8f9b8686397a47409e5d4b2d855191a2 => generated 283 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:47.704414 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decd941f-ec91-4d22-b038-e9fbe500a996 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:47.730872 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:47.746856 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6a40dd-dcf1-4efa-814b-7a9251bf2ef9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:47.750517 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6a40dd-dcf1-4efa-814b-7a9251bf2ef9 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:47.829250 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decd941f-ec91-4d22-b038-e9fbe500a996 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:47.840651 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:47.868198 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 112/226] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:00:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 12:00:47.873428 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6a40dd-dcf1-4efa-814b-7a9251bf2ef9 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:47.874699 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b941e3-3aab-4be1-bbd9-088ee12bb7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:47.875326 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b941e3-3aab-4be1-bbd9-088ee12bb7bc None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:47.875604 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b941e3-3aab-4be1-bbd9-088ee12bb7bc None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:47.875873 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b941e3-3aab-4be1-bbd9-088ee12bb7bc None admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:47.876540 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b941e3-3aab-4be1-bbd9-088ee12bb7bc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:00:47.876837 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b941e3-3aab-4be1-bbd9-088ee12bb7bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:47.878208 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b941e3-3aab-4be1-bbd9-088ee12bb7bc None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:47.892574 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 115/227] 10.222.0.61 () {56 vars in 1111 bytes} [Fri Aug 26 12:00:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:47.893501 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b941e3-3aab-4be1-bbd9-088ee12bb7bc None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:47.968616 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:47.978623 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6a40dd-dcf1-4efa-814b-7a9251bf2ef9 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:47.995148 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b941e3-3aab-4be1-bbd9-088ee12bb7bc None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:48.017617 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e44cd1-9360-48a5-b0bc-73a6eaf6afca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:48.018580 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e44cd1-9360-48a5-b0bc-73a6eaf6afca None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:48.019056 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e44cd1-9360-48a5-b0bc-73a6eaf6afca None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:48.019600 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e44cd1-9360-48a5-b0bc-73a6eaf6afca None admin] PATH_INFO: `/v3/projects/16904d4c978c4c12abd64d625faa3eda` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:48.021054 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e44cd1-9360-48a5-b0bc-73a6eaf6afca None admin] RBAC: Authorizing `identity:get_project(project_id=16904d4c978c4c12abd64d625faa3eda)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:48.022839 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e44cd1-9360-48a5-b0bc-73a6eaf6afca None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:48.025223 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 113/228] 10.222.0.61 () {56 vars in 1186 bytes} [Fri Aug 26 12:00:47 2022] GET /identity/v3/projects/16904d4c978c4c12abd64d625faa3eda => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:48.026421 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e44cd1-9360-48a5-b0bc-73a6eaf6afca None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:48.031804 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8c4b87-1434-4309-a960-8a52b573b56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:48.031983 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c4b87-1434-4309-a960-8a52b573b56d None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:48.032260 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c4b87-1434-4309-a960-8a52b573b56d None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:48.032528 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c4b87-1434-4309-a960-8a52b573b56d None admin] PATH_INFO: `/v3/projects/16904d4c978c4c12abd64d625faa3eda` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:48.033645 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8c4b87-1434-4309-a960-8a52b573b56d None admin] RBAC: Authorizing `identity:get_project(project_id=16904d4c978c4c12abd64d625faa3eda)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:48.035075 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8c4b87-1434-4309-a960-8a52b573b56d None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:48.041587 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 116/229] 10.222.0.61 () {56 vars in 1186 bytes} [Fri Aug 26 12:00:47 2022] GET /identity/v3/projects/16904d4c978c4c12abd64d625faa3eda => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:48.041587 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8c4b87-1434-4309-a960-8a52b573b56d None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:48.078730 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:48.140113 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8c4b87-1434-4309-a960-8a52b573b56d None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:48.145789 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e44cd1-9360-48a5-b0bc-73a6eaf6afca None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:48.174970 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e4f0b8-8056-4253-a87a-18af7deb2093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:48.175658 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4f0b8-8056-4253-a87a-18af7deb2093 None admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:48.176075 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4f0b8-8056-4253-a87a-18af7deb2093 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:48.176482 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4f0b8-8056-4253-a87a-18af7deb2093 None admin] PATH_INFO: `/v3/projects/16904d4c978c4c12abd64d625faa3eda/users/8f9b8686397a47409e5d4b2d855191a2/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:48.178884 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e4f0b8-8056-4253-a87a-18af7deb2093 None admin] RBAC: Authorizing `identity:create_grant(project_id=16904d4c978c4c12abd64d625faa3eda, user_id=8f9b8686397a47409e5d4b2d855191a2, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:48.180714 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e4f0b8-8056-4253-a87a-18af7deb2093 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:48.183877 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cdc861-8ed9-48c0-840c-aeb4d61c8287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:48.184858 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cdc861-8ed9-48c0-840c-aeb4d61c8287 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:48.185244 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cdc861-8ed9-48c0-840c-aeb4d61c8287 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:48.186063 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cdc861-8ed9-48c0-840c-aeb4d61c8287 None admin] PATH_INFO: `/v3/users/8f9b8686397a47409e5d4b2d855191a2` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:48.187474 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cdc861-8ed9-48c0-840c-aeb4d61c8287 None admin] RBAC: Authorizing `identity:get_user(user_id=8f9b8686397a47409e5d4b2d855191a2)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:48.189076 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cdc861-8ed9-48c0-840c-aeb4d61c8287 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:48.195270 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 114/230] 10.222.0.61 () {56 vars in 1177 bytes} [Fri Aug 26 12:00:48 2022] GET /identity/v3/users/8f9b8686397a47409e5d4b2d855191a2 => generated 283 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:48.195270 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cdc861-8ed9-48c0-840c-aeb4d61c8287 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:48.202053 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 117/231] 10.222.0.61 () {58 vars in 1439 bytes} [Fri Aug 26 12:00:48 2022] PUT /identity/v3/projects/16904d4c978c4c12abd64d625faa3eda/users/8f9b8686397a47409e5d4b2d855191a2/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:00:48.203307 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e4f0b8-8056-4253-a87a-18af7deb2093 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:48.248919 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:48.330179 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:48.334956 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e4f0b8-8056-4253-a87a-18af7deb2093 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:48.386546 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b78a15-e9b7-40fe-b578-6b9ccd1839e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:48.387195 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b78a15-e9b7-40fe-b578-6b9ccd1839e1 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:48.387519 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b78a15-e9b7-40fe-b578-6b9ccd1839e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:48.387798 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b78a15-e9b7-40fe-b578-6b9ccd1839e1 None admin] PATH_INFO: `/v3/projects/16904d4c978c4c12abd64d625faa3eda` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:48.389167 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b78a15-e9b7-40fe-b578-6b9ccd1839e1 None admin] RBAC: Authorizing `identity:get_project(project_id=16904d4c978c4c12abd64d625faa3eda)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:48.390672 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b78a15-e9b7-40fe-b578-6b9ccd1839e1 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:48.392483 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 118/232] 10.222.0.61 () {56 vars in 1186 bytes} [Fri Aug 26 12:00:48 2022] GET /identity/v3/projects/16904d4c978c4c12abd64d625faa3eda => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:48.398794 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b78a15-e9b7-40fe-b578-6b9ccd1839e1 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:48.400736 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cdc861-8ed9-48c0-840c-aeb4d61c8287 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:48.453829 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b34d70-3528-48ae-8242-0aee0f33e9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:48.455247 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b34d70-3528-48ae-8242-0aee0f33e9a7 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:48.455760 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b34d70-3528-48ae-8242-0aee0f33e9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:48.456103 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b34d70-3528-48ae-8242-0aee0f33e9a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:48.456906 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b34d70-3528-48ae-8242-0aee0f33e9a7 None admin] RBAC: Adding query filter params (role.id=b2403690f5ad443aad059f5ee024f61c, scope.project.id=16904d4c978c4c12abd64d625faa3eda, user.id=cb0932d41804458783f0b2063f06bd69) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:00:48.457279 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b34d70-3528-48ae-8242-0aee0f33e9a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:48.459597 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b34d70-3528-48ae-8242-0aee0f33e9a7 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:48.468200 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 115/233] 10.222.0.61 () {56 vars in 1374 bytes} [Fri Aug 26 12:00:48 2022] GET /identity/v3/role_assignments?user.id=cb0932d41804458783f0b2063f06bd69&scope.project.id=16904d4c978c4c12abd64d625faa3eda&role.id=b2403690f5ad443aad059f5ee024f61c => generated 259 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:48.525852 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b78a15-e9b7-40fe-b578-6b9ccd1839e1 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:48.566199 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985ff1e9-f70c-4b9c-b54e-4ed7f5fc109c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:48.567188 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ff1e9-f70c-4b9c-b54e-4ed7f5fc109c None admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:48.567635 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ff1e9-f70c-4b9c-b54e-4ed7f5fc109c None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:48.568082 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ff1e9-f70c-4b9c-b54e-4ed7f5fc109c None admin] PATH_INFO: `/v3/projects/16904d4c978c4c12abd64d625faa3eda/users/8f9b8686397a47409e5d4b2d855191a2/roles/a60f381814fe4c1da324d13c8e7d5d20` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:48.570434 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985ff1e9-f70c-4b9c-b54e-4ed7f5fc109c None admin] RBAC: Authorizing `identity:create_grant(project_id=16904d4c978c4c12abd64d625faa3eda, user_id=8f9b8686397a47409e5d4b2d855191a2, role_id=a60f381814fe4c1da324d13c8e7d5d20)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:48.572153 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985ff1e9-f70c-4b9c-b54e-4ed7f5fc109c None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:48.600073 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 119/234] 10.222.0.61 () {58 vars in 1439 bytes} [Fri Aug 26 12:00:48 2022] PUT /identity/v3/projects/16904d4c978c4c12abd64d625faa3eda/users/8f9b8686397a47409e5d4b2d855191a2/roles/a60f381814fe4c1da324d13c8e7d5d20 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:00:49.478286 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e00026-3405-4a27-aaf8-1b58293b7274 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:49.478791 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e00026-3405-4a27-aaf8-1b58293b7274 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:49.478866 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e00026-3405-4a27-aaf8-1b58293b7274 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:49.480317 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 116/235] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:00:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:00:49.485957 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6427b-054e-4439-b062-4197f965d963 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:49.486371 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6427b-054e-4439-b062-4197f965d963 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:49.486538 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6427b-054e-4439-b062-4197f965d963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:49.508798 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-17a6427b-054e-4439-b062-4197f965d963 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:00:49.509964 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-17a6427b-054e-4439-b062-4197f965d963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:00:49.535902 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:49.587620 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 120/236] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:00:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:00:49.620554 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ae531a-92bc-40b4-a424-f92a14b2cd92 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:49.620866 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ae531a-92bc-40b4-a424-f92a14b2cd92 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:49.621097 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ae531a-92bc-40b4-a424-f92a14b2cd92 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:49.622435 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 117/237] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:00:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:00:49.628051 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f9b79-ce8d-4459-9797-3ed611b09947 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:49.628337 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f9b79-ce8d-4459-9797-3ed611b09947 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:49.628609 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f9b79-ce8d-4459-9797-3ed611b09947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:49.651113 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-d44f9b79-ce8d-4459-9797-3ed611b09947 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:00:49.652291 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-d44f9b79-ce8d-4459-9797-3ed611b09947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:00:49.745415 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 121/238] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:00:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:00:49.941987 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ae531a-92bc-40b4-a424-f92a14b2cd92 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:50.047789 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:50.063891 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ae531a-92bc-40b4-a424-f92a14b2cd92 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:50.109416 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc4b57c-8849-4966-adf5-8f697e096324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:50.110052 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc4b57c-8849-4966-adf5-8f697e096324 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:50.110329 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc4b57c-8849-4966-adf5-8f697e096324 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:50.110600 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc4b57c-8849-4966-adf5-8f697e096324 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:50.127573 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc4b57c-8849-4966-adf5-8f697e096324 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:00:50.132658 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-ddc4b57c-8849-4966-adf5-8f697e096324 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 12:00:50.132658 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:00:50.132658 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:00:50.132658 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:00:50.132658 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:00:50.132658 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:00:50.132658 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:00:50.132658 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:00:50.132658 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:00:50.132658 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:00:50.132658 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:00:50.132658 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:00:50.132658 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:00:50.132658 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:00:50.132658 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:00:50.132658 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:00:50.133779 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:00:50.133779 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:00:50.133779 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:00:50.133779 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:00:50.133779 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:00:50.133779 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:00:50.133779 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:00:50.133779 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:00:50.133779 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:00:50.133779 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:00:50.133779 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:00:50.133779 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:00:50.133779 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:00:50.133779 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:00:50.133779 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:00:50.133779 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:00:50.134569 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:00:50.134569 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:00:50.134569 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:00:50.134569 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:00:50.134569 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:00:50.134569 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:00:50.134569 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:00:50.134569 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:00:50.134569 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:00:50.134569 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:00:50.134569 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 12:00:50.134569 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:00:50.134569 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 118/239] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:00:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:00:50.144341 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44f9b79-ce8d-4459-9797-3ed611b09947 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:50.178451 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc4b57c-8849-4966-adf5-8f697e096324 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:50.290200 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:50.317769 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44f9b79-ce8d-4459-9797-3ed611b09947 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:50.356738 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc4b57c-8849-4966-adf5-8f697e096324 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:50.377066 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce762e16-826b-49c9-84f6-3819a2124490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:50.377675 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce762e16-826b-49c9-84f6-3819a2124490 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:50.377951 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce762e16-826b-49c9-84f6-3819a2124490 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:50.378220 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce762e16-826b-49c9-84f6-3819a2124490 None admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:50.378889 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce762e16-826b-49c9-84f6-3819a2124490 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:00:50.379191 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce762e16-826b-49c9-84f6-3819a2124490 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:50.380603 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce762e16-826b-49c9-84f6-3819a2124490 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:50.397865 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 122/240] 10.222.0.61 () {56 vars in 1101 bytes} [Fri Aug 26 12:00:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:50.402193 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce762e16-826b-49c9-84f6-3819a2124490 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:50.418269 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1866efc-8c87-4e48-a35d-c55e10e9ba4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:50.418965 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1866efc-8c87-4e48-a35d-c55e10e9ba4f None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:50.419274 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1866efc-8c87-4e48-a35d-c55e10e9ba4f None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:50.419549 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1866efc-8c87-4e48-a35d-c55e10e9ba4f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:50.421973 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1866efc-8c87-4e48-a35d-c55e10e9ba4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:50.423538 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1866efc-8c87-4e48-a35d-c55e10e9ba4f None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:50.426489 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 119/241] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:00:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:50.434227 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1866efc-8c87-4e48-a35d-c55e10e9ba4f None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:50.473474 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:50.595505 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1866efc-8c87-4e48-a35d-c55e10e9ba4f None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:50.597236 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce762e16-826b-49c9-84f6-3819a2124490 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:50.635593 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5f94da-7e3c-4d2a-9906-43f30c72dab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:50.636319 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5f94da-7e3c-4d2a-9906-43f30c72dab8 None admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:50.636953 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5f94da-7e3c-4d2a-9906-43f30c72dab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:50.637232 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5f94da-7e3c-4d2a-9906-43f30c72dab8 None admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:50.638218 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5f94da-7e3c-4d2a-9906-43f30c72dab8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:50.640624 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5f94da-7e3c-4d2a-9906-43f30c72dab8 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:50.642214 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e6a9ba-98d5-4a6d-b547-7e7ec6dfd0a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:50.642998 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e6a9ba-98d5-4a6d-b547-7e7ec6dfd0a0 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:50.643445 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e6a9ba-98d5-4a6d-b547-7e7ec6dfd0a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:50.643875 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e6a9ba-98d5-4a6d-b547-7e7ec6dfd0a0 None admin] PATH_INFO: `/v3/users/2007314f4a594b7a8bdf4b3932c9bbab` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:50.664072 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e6a9ba-98d5-4a6d-b547-7e7ec6dfd0a0 None admin] RBAC: Authorizing `identity:get_user(user_id=2007314f4a594b7a8bdf4b3932c9bbab)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:50.665656 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e6a9ba-98d5-4a6d-b547-7e7ec6dfd0a0 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:50.672037 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 120/243] 10.222.0.61 () {60 vars in 1132 bytes} [Fri Aug 26 12:00:50 2022] POST /identity/v3/users => generated 293 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:00:50.672337 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 123/243] 10.222.0.61 () {56 vars in 1177 bytes} [Fri Aug 26 12:00:50 2022] GET /identity/v3/users/2007314f4a594b7a8bdf4b3932c9bbab => generated 297 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:50.679267 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e6a9ba-98d5-4a6d-b547-7e7ec6dfd0a0 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:50.680156 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b57fbe-6c92-41d9-8f6d-19a70a2003f0 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:50.681759 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b57fbe-6c92-41d9-8f6d-19a70a2003f0 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:50.682251 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b57fbe-6c92-41d9-8f6d-19a70a2003f0 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:50.686357 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 121/244] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:00:50 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:00:50.696726 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afff245-5c98-465d-97b7-237c1062ff1f None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:50.697378 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afff245-5c98-465d-97b7-237c1062ff1f None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:50.697877 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afff245-5c98-465d-97b7-237c1062ff1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:50.705169 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:50.738542 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-2afff245-5c98-465d-97b7-237c1062ff1f None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:00:50.742731 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-2afff245-5c98-465d-97b7-237c1062ff1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:00:50.879347 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 122/245] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:00:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 12:00:50.879539 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e6a9ba-98d5-4a6d-b547-7e7ec6dfd0a0 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:50.880773 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518db2de-79b2-45f2-879a-a465fb391ea4 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:50.881176 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518db2de-79b2-45f2-879a-a465fb391ea4 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:50.881454 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518db2de-79b2-45f2-879a-a465fb391ea4 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:50.882912 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 123/246] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:00:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:00:50.894282 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981ec255-66f9-457b-a9cb-4c2b73c135ef None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:50.894562 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981ec255-66f9-457b-a9cb-4c2b73c135ef None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:50.894845 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981ec255-66f9-457b-a9cb-4c2b73c135ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:50.906640 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:50.925659 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-981ec255-66f9-457b-a9cb-4c2b73c135ef None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:00:50.928499 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-981ec255-66f9-457b-a9cb-4c2b73c135ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:00:50.935073 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3ab8b3-8d72-4c55-99da-8b6a1b7cec28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:50.935760 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3ab8b3-8d72-4c55-99da-8b6a1b7cec28 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:50.936042 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3ab8b3-8d72-4c55-99da-8b6a1b7cec28 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:50.936319 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3ab8b3-8d72-4c55-99da-8b6a1b7cec28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:51.021614 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 124/247] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:00:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:00:51.039720 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3ab8b3-8d72-4c55-99da-8b6a1b7cec28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:51.041641 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3ab8b3-8d72-4c55-99da-8b6a1b7cec28 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:51.051143 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:51.104617 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc21694-571e-49e8-8eab-103093bac7b0 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:51.104924 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc21694-571e-49e8-8eab-103093bac7b0 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:51.105169 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc21694-571e-49e8-8eab-103093bac7b0 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:51.106510 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 125/248] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:00:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 12:00:51.112133 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2078ee4-3739-446a-a917-6facc004018b None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:51.112708 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2078ee4-3739-446a-a917-6facc004018b None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:51.113174 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2078ee4-3739-446a-a917-6facc004018b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:51.136318 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-e2078ee4-3739-446a-a917-6facc004018b None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:00:51.137803 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-e2078ee4-3739-446a-a917-6facc004018b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:00:51.161911 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 124/249] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:00:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 26 12:00:51.168277 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3ab8b3-8d72-4c55-99da-8b6a1b7cec28 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:51.223026 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 126/250] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:00:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:00:51.231227 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2078ee4-3739-446a-a917-6facc004018b None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:51.310160 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3ab8b3-8d72-4c55-99da-8b6a1b7cec28 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:51.322538 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:51.325143 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:51.372552 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2078ee4-3739-446a-a917-6facc004018b None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:51.385245 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6832233d-d972-45cb-a0d0-6cccbb37fa96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:51.385903 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6832233d-d972-45cb-a0d0-6cccbb37fa96 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:51.386165 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6832233d-d972-45cb-a0d0-6cccbb37fa96 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:51.386497 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6832233d-d972-45cb-a0d0-6cccbb37fa96 None admin] PATH_INFO: `/v3/projects/16904d4c978c4c12abd64d625faa3eda` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:51.387608 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6832233d-d972-45cb-a0d0-6cccbb37fa96 None admin] RBAC: Authorizing `identity:get_project(project_id=16904d4c978c4c12abd64d625faa3eda)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:51.389763 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6832233d-d972-45cb-a0d0-6cccbb37fa96 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:51.391756 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 125/251] 10.222.0.61 () {56 vars in 1186 bytes} [Fri Aug 26 12:00:51 2022] GET /identity/v3/projects/16904d4c978c4c12abd64d625faa3eda => generated 302 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:51.392606 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6832233d-d972-45cb-a0d0-6cccbb37fa96 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:51.417994 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeefd4e7-4deb-456d-a119-3f0a9f33f421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:51.418632 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeefd4e7-4deb-456d-a119-3f0a9f33f421 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:51.418912 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeefd4e7-4deb-456d-a119-3f0a9f33f421 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:51.419324 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeefd4e7-4deb-456d-a119-3f0a9f33f421 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:51.430550 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeefd4e7-4deb-456d-a119-3f0a9f33f421 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:00:51.435898 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-eeefd4e7-4deb-456d-a119-3f0a9f33f421 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 12:00:51.435898 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:00:51.435898 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:00:51.435898 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:00:51.435898 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:00:51.435898 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:00:51.435898 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:00:51.435898 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:00:51.435898 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:00:51.435898 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:00:51.435898 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:00:51.435898 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:00:51.435898 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:00:51.435898 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:00:51.435898 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:00:51.435898 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:00:51.436797 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:00:51.436797 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:00:51.436797 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:00:51.436797 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:00:51.436797 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:00:51.436797 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:00:51.436797 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:00:51.436797 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:00:51.436797 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:00:51.436797 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:00:51.436797 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:00:51.436797 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:00:51.436797 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:00:51.436797 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:00:51.436797 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:00:51.436797 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:00:51.437583 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:00:51.437583 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:00:51.437583 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:00:51.437583 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:00:51.437583 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:00:51.437583 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:00:51.437583 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:00:51.437583 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:00:51.437583 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:00:51.437583 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:00:51.437583 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 12:00:51.437583 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:00:51.438177 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 127/252] 10.222.0.61 () {56 vars in 1096 bytes} [Fri Aug 26 12:00:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:00:51.438720 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeefd4e7-4deb-456d-a119-3f0a9f33f421 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:51.564613 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6832233d-d972-45cb-a0d0-6cccbb37fa96 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:51.591310 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeefd4e7-4deb-456d-a119-3f0a9f33f421 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:51.619685 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:51.621131 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c0c14e-9093-49c6-84ca-3bf685eb2a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:51.621773 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c0c14e-9093-49c6-84ca-3bf685eb2a8e None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:51.622057 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c0c14e-9093-49c6-84ca-3bf685eb2a8e None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:51.622332 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c0c14e-9093-49c6-84ca-3bf685eb2a8e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:51.628569 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b60aa06-9474-4535-9c19-18199a09f5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:51.629173 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b60aa06-9474-4535-9c19-18199a09f5da None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:51.629582 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b60aa06-9474-4535-9c19-18199a09f5da None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:51.629857 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b60aa06-9474-4535-9c19-18199a09f5da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:51.630649 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b60aa06-9474-4535-9c19-18199a09f5da None admin] RBAC: Adding query filter params (role.id=12a2a8379bd241a098646f1c8d881614, scope.project.id=16904d4c978c4c12abd64d625faa3eda, user.id=2007314f4a594b7a8bdf4b3932c9bbab) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:00:51.630963 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b60aa06-9474-4535-9c19-18199a09f5da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:51.632522 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b60aa06-9474-4535-9c19-18199a09f5da None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:51.634283 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c0c14e-9093-49c6-84ca-3bf685eb2a8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:00:51.639503 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 128/253] 10.222.0.61 () {56 vars in 1374 bytes} [Fri Aug 26 12:00:51 2022] GET /identity/v3/role_assignments?user.id=2007314f4a594b7a8bdf4b3932c9bbab&scope.project.id=16904d4c978c4c12abd64d625faa3eda&role.id=12a2a8379bd241a098646f1c8d881614 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:51.640229 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b60aa06-9474-4535-9c19-18199a09f5da None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:51.640934 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-e5c0c14e-9093-49c6-84ca-3bf685eb2a8e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 12:00:51.640934 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:00:51.640934 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:00:51.640934 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:00:51.640934 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:00:51.640934 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:00:51.640934 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:00:51.640934 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:00:51.640934 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:00:51.640934 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:00:51.640934 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:00:51.640934 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:00:51.640934 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:00:51.640934 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:00:51.640934 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:00:51.640934 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:00:51.641801 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:00:51.641801 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:00:51.641801 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:00:51.641801 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:00:51.641801 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:00:51.641801 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:00:51.641801 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:00:51.641801 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:00:51.641801 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:00:51.641801 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:00:51.641801 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:00:51.641801 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:00:51.641801 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:00:51.641801 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:00:51.641801 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:00:51.641801 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:00:51.642667 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:00:51.642667 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:00:51.642667 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:00:51.642667 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:00:51.642667 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:00:51.642667 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:00:51.642667 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:00:51.642667 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:00:51.642667 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:00:51.642667 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:00:51.642667 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 12:00:51.642667 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:00:51.645191 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 126/254] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:00:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:00:51.651082 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c0c14e-9093-49c6-84ca-3bf685eb2a8e None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:51.691441 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:51.751456 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b60aa06-9474-4535-9c19-18199a09f5da None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:51.772663 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c0c14e-9093-49c6-84ca-3bf685eb2a8e None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:51.788648 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7df941-5816-47df-9e65-9b6ee4242812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:51.789278 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7df941-5816-47df-9e65-9b6ee4242812 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:51.789559 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7df941-5816-47df-9e65-9b6ee4242812 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:51.789833 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7df941-5816-47df-9e65-9b6ee4242812 None admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:51.790527 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7df941-5816-47df-9e65-9b6ee4242812 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:00:51.790831 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7df941-5816-47df-9e65-9b6ee4242812 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:51.792295 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7df941-5816-47df-9e65-9b6ee4242812 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:51.802679 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 129/255] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:00:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:51.803639 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7df941-5816-47df-9e65-9b6ee4242812 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:51.842787 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f1471f-c6fc-4345-a2ee-f4c553c27503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:51.843478 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f1471f-c6fc-4345-a2ee-f4c553c27503 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:51.843765 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f1471f-c6fc-4345-a2ee-f4c553c27503 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:51.844044 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f1471f-c6fc-4345-a2ee-f4c553c27503 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:51.859268 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:51.875659 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f1471f-c6fc-4345-a2ee-f4c553c27503 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:00:51.880627 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-38f1471f-c6fc-4345-a2ee-f4c553c27503 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 12:00:51.880627 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:00:51.880627 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:00:51.880627 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:00:51.880627 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:00:51.880627 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:00:51.880627 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:00:51.880627 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:00:51.880627 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:00:51.880627 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:00:51.880627 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:00:51.880627 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:00:51.880627 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:00:51.880627 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:00:51.880627 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:00:51.880627 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:00:51.881674 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:00:51.881674 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:00:51.881674 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:00:51.881674 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:00:51.881674 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:00:51.881674 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:00:51.881674 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:00:51.881674 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:00:51.881674 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:00:51.881674 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:00:51.881674 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:00:51.881674 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:00:51.881674 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:00:51.881674 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:00:51.881674 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:00:51.881674 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:00:51.882490 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:00:51.882490 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:00:51.882490 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:00:51.882490 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:00:51.882490 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:00:51.882490 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:00:51.882490 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:00:51.882490 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:00:51.882490 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:00:51.882490 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:00:51.882490 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 12:00:51.882490 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:00:51.882490 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 127/256] 10.222.0.61 () {56 vars in 1114 bytes} [Fri Aug 26 12:00:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:00:51.883433 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f1471f-c6fc-4345-a2ee-f4c553c27503 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:51.932001 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7df941-5816-47df-9e65-9b6ee4242812 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:51.994252 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09df3a8d-51f6-4e4e-a706-7b687024967f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:51.994871 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df3a8d-51f6-4e4e-a706-7b687024967f None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:51.995151 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df3a8d-51f6-4e4e-a706-7b687024967f None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:51.995453 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df3a8d-51f6-4e4e-a706-7b687024967f None admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:51.996120 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09df3a8d-51f6-4e4e-a706-7b687024967f None admin] RBAC: Adding query filter params (name=member) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:00:51.996422 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09df3a8d-51f6-4e4e-a706-7b687024967f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:51.998319 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09df3a8d-51f6-4e4e-a706-7b687024967f None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:52.017882 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 130/257] 10.222.0.61 () {56 vars in 1101 bytes} [Fri Aug 26 12:00:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:52.018741 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09df3a8d-51f6-4e4e-a706-7b687024967f None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:52.063654 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:52.073219 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f1471f-c6fc-4345-a2ee-f4c553c27503 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:52.125500 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68140873-4b85-4203-9d21-3cb19091010a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:52.126144 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68140873-4b85-4203-9d21-3cb19091010a None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:52.126431 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68140873-4b85-4203-9d21-3cb19091010a None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:52.126763 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68140873-4b85-4203-9d21-3cb19091010a None admin] PATH_INFO: `/v3/users/8f9b8686397a47409e5d4b2d855191a2` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:52.128400 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68140873-4b85-4203-9d21-3cb19091010a None admin] RBAC: Authorizing `identity:get_user(user_id=8f9b8686397a47409e5d4b2d855191a2)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:52.130006 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68140873-4b85-4203-9d21-3cb19091010a None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:52.136478 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 128/258] 10.222.0.61 () {56 vars in 1177 bytes} [Fri Aug 26 12:00:51 2022] GET /identity/v3/users/8f9b8686397a47409e5d4b2d855191a2 => generated 283 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:52.137379 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68140873-4b85-4203-9d21-3cb19091010a None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:52.249317 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09df3a8d-51f6-4e4e-a706-7b687024967f None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:52.256355 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68140873-4b85-4203-9d21-3cb19091010a None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:52.266183 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:52.298128 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4120934-9200-4dbd-a0ed-848f5763826d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:52.299115 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4120934-9200-4dbd-a0ed-848f5763826d None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:52.299724 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4120934-9200-4dbd-a0ed-848f5763826d None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:52.300194 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4120934-9200-4dbd-a0ed-848f5763826d None admin] PATH_INFO: `/v3/users/cb0932d41804458783f0b2063f06bd69` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:52.301531 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4120934-9200-4dbd-a0ed-848f5763826d None admin] RBAC: Authorizing `identity:get_user(user_id=cb0932d41804458783f0b2063f06bd69)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:52.303521 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4120934-9200-4dbd-a0ed-848f5763826d None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:52.305358 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 129/259] 10.222.0.61 () {56 vars in 1177 bytes} [Fri Aug 26 12:00:52 2022] GET /identity/v3/users/cb0932d41804458783f0b2063f06bd69 => generated 297 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:52.306427 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4120934-9200-4dbd-a0ed-848f5763826d None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:52.307900 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e978b11-d7c6-4b83-bcf8-d6dc23b1e1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:52.308529 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e978b11-d7c6-4b83-bcf8-d6dc23b1e1ac None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:52.308816 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e978b11-d7c6-4b83-bcf8-d6dc23b1e1ac None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:52.309096 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e978b11-d7c6-4b83-bcf8-d6dc23b1e1ac None admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:52.309775 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e978b11-d7c6-4b83-bcf8-d6dc23b1e1ac None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:00:52.310080 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e978b11-d7c6-4b83-bcf8-d6dc23b1e1ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:52.312852 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e978b11-d7c6-4b83-bcf8-d6dc23b1e1ac None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:52.334657 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 131/260] 10.222.0.61 () {56 vars in 1111 bytes} [Fri Aug 26 12:00:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:52.334657 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e978b11-d7c6-4b83-bcf8-d6dc23b1e1ac None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:52.348293 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:52.428949 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4120934-9200-4dbd-a0ed-848f5763826d None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:52.469067 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7c78e8-b543-4d66-8d6b-b405fc64cfd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:52.470020 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c78e8-b543-4d66-8d6b-b405fc64cfd0 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:52.470502 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c78e8-b543-4d66-8d6b-b405fc64cfd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:52.471004 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c78e8-b543-4d66-8d6b-b405fc64cfd0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:52.483763 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e978b11-d7c6-4b83-bcf8-d6dc23b1e1ac None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:52.530597 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b5648f-4a4a-44bc-af9f-4bb84a412d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:52.531154 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b5648f-4a4a-44bc-af9f-4bb84a412d8d None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:52.531531 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b5648f-4a4a-44bc-af9f-4bb84a412d8d None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:52.531806 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b5648f-4a4a-44bc-af9f-4bb84a412d8d None admin] PATH_INFO: `/v3/projects/16904d4c978c4c12abd64d625faa3eda` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:52.533127 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b5648f-4a4a-44bc-af9f-4bb84a412d8d None admin] RBAC: Authorizing `identity:get_project(project_id=16904d4c978c4c12abd64d625faa3eda)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:52.534613 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b5648f-4a4a-44bc-af9f-4bb84a412d8d None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:52.536414 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 132/261] 10.222.0.61 () {56 vars in 1186 bytes} [Fri Aug 26 12:00:52 2022] GET /identity/v3/projects/16904d4c978c4c12abd64d625faa3eda => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:52.537233 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b5648f-4a4a-44bc-af9f-4bb84a412d8d None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:52.556620 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7c78e8-b543-4d66-8d6b-b405fc64cfd0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:52.558445 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7c78e8-b543-4d66-8d6b-b405fc64cfd0 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:52.590843 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:52.631799 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:52.674488 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 130/262] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:00:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 12:00:52.680524 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7c78e8-b543-4d66-8d6b-b405fc64cfd0 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:52.732379 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b5648f-4a4a-44bc-af9f-4bb84a412d8d None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:52.791233 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7c78e8-b543-4d66-8d6b-b405fc64cfd0 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:52.823767 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:52.827239 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a220bf-f85f-490d-9042-6561e01b67c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:52.827938 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a220bf-f85f-490d-9042-6561e01b67c7 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:52.828194 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a220bf-f85f-490d-9042-6561e01b67c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:52.828508 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a220bf-f85f-490d-9042-6561e01b67c7 None admin] PATH_INFO: `/v3/users/8f9b8686397a47409e5d4b2d855191a2` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:52.829741 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a220bf-f85f-490d-9042-6561e01b67c7 None admin] RBAC: Authorizing `identity:get_user(user_id=8f9b8686397a47409e5d4b2d855191a2)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:52.831182 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a220bf-f85f-490d-9042-6561e01b67c7 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:52.833916 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362f3a46-dc1d-4521-91c6-789774e1c93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:52.834825 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f3a46-dc1d-4521-91c6-789774e1c93d None admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:52.835353 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f3a46-dc1d-4521-91c6-789774e1c93d None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:52.835830 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f3a46-dc1d-4521-91c6-789774e1c93d None admin] PATH_INFO: `/v3/projects/16904d4c978c4c12abd64d625faa3eda/users/cb0932d41804458783f0b2063f06bd69/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:52.838213 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362f3a46-dc1d-4521-91c6-789774e1c93d None admin] RBAC: Authorizing `identity:create_grant(project_id=16904d4c978c4c12abd64d625faa3eda, user_id=cb0932d41804458783f0b2063f06bd69, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:52.839896 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362f3a46-dc1d-4521-91c6-789774e1c93d None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:52.845089 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 133/263] 10.222.0.61 () {56 vars in 1177 bytes} [Fri Aug 26 12:00:52 2022] GET /identity/v3/users/8f9b8686397a47409e5d4b2d855191a2 => generated 283 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:52.845089 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a220bf-f85f-490d-9042-6561e01b67c7 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:52.869988 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 131/264] 10.222.0.61 () {58 vars in 1439 bytes} [Fri Aug 26 12:00:52 2022] PUT /identity/v3/projects/16904d4c978c4c12abd64d625faa3eda/users/cb0932d41804458783f0b2063f06bd69/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:00:52.874282 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362f3a46-dc1d-4521-91c6-789774e1c93d None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:53.056078 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362f3a46-dc1d-4521-91c6-789774e1c93d None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:53.093821 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a220bf-f85f-490d-9042-6561e01b67c7 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:53.098958 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fd0f87-22b8-4ec8-a664-ceed1fbd1cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:53.099916 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd0f87-22b8-4ec8-a664-ceed1fbd1cab None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:53.100401 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd0f87-22b8-4ec8-a664-ceed1fbd1cab None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:53.100896 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd0f87-22b8-4ec8-a664-ceed1fbd1cab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:53.117110 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:53.173436 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:53.188769 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fd0f87-22b8-4ec8-a664-ceed1fbd1cab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:53.190558 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fd0f87-22b8-4ec8-a664-ceed1fbd1cab None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:53.191907 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb352df-2ca5-4705-85ae-a2c0becc2dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:53.192560 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb352df-2ca5-4705-85ae-a2c0becc2dd8 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:53.192841 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb352df-2ca5-4705-85ae-a2c0becc2dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:53.193109 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb352df-2ca5-4705-85ae-a2c0becc2dd8 None admin] PATH_INFO: `/v3/projects/16904d4c978c4c12abd64d625faa3eda` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:53.194276 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb352df-2ca5-4705-85ae-a2c0becc2dd8 None admin] RBAC: Authorizing `identity:get_project(project_id=16904d4c978c4c12abd64d625faa3eda)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:53.195735 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb352df-2ca5-4705-85ae-a2c0becc2dd8 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:53.200628 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 134/265] 10.222.0.61 () {56 vars in 1186 bytes} [Fri Aug 26 12:00:52 2022] GET /identity/v3/projects/16904d4c978c4c12abd64d625faa3eda => generated 302 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:53.204941 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb352df-2ca5-4705-85ae-a2c0becc2dd8 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:53.338615 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb352df-2ca5-4705-85ae-a2c0becc2dd8 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:53.414296 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d110635b-92b5-4f3c-829c-f2be4de21246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:53.415350 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d110635b-92b5-4f3c-829c-f2be4de21246 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:53.415834 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d110635b-92b5-4f3c-829c-f2be4de21246 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:53.416306 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d110635b-92b5-4f3c-829c-f2be4de21246 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:53.417236 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d110635b-92b5-4f3c-829c-f2be4de21246 None admin] RBAC: Adding query filter params (role.id=70ec1ba484714c8ca3adb8fe5845107c, scope.project.id=16904d4c978c4c12abd64d625faa3eda, user.id=8f9b8686397a47409e5d4b2d855191a2) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:00:53.417805 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d110635b-92b5-4f3c-829c-f2be4de21246 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:53.419574 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d110635b-92b5-4f3c-829c-f2be4de21246 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:53.429910 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 132/266] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:00:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 12:00:53.434238 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 135/267] 10.222.0.61 () {56 vars in 1374 bytes} [Fri Aug 26 12:00:53 2022] GET /identity/v3/role_assignments?user.id=8f9b8686397a47409e5d4b2d855191a2&scope.project.id=16904d4c978c4c12abd64d625faa3eda&role.id=70ec1ba484714c8ca3adb8fe5845107c => generated 608 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:00:53.449522 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fd0f87-22b8-4ec8-a664-ceed1fbd1cab None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:53.565143 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:53.575644 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fd0f87-22b8-4ec8-a664-ceed1fbd1cab None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:53.596497 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6cb779-8aa0-42e0-8133-c8b8f8ebb307 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:53.597093 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6cb779-8aa0-42e0-8133-c8b8f8ebb307 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:53.597572 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6cb779-8aa0-42e0-8133-c8b8f8ebb307 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:53.599284 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 136/268] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:00:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 12:00:53.604452 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c1b427-e64c-4274-acb9-6774a405ec47 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:53.605056 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c1b427-e64c-4274-acb9-6774a405ec47 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:53.605532 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c1b427-e64c-4274-acb9-6774a405ec47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:53.627036 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fe2631-393b-4ba0-91be-207ee42d863c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:53.631093 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe2631-393b-4ba0-91be-207ee42d863c None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:53.631416 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe2631-393b-4ba0-91be-207ee42d863c None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:53.631701 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe2631-393b-4ba0-91be-207ee42d863c None admin] PATH_INFO: `/v3/projects/16904d4c978c4c12abd64d625faa3eda` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:53.634127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fe2631-393b-4ba0-91be-207ee42d863c None admin] RBAC: Authorizing `identity:get_project(project_id=16904d4c978c4c12abd64d625faa3eda)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:53.640169 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fe2631-393b-4ba0-91be-207ee42d863c None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:53.641991 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 133/269] 10.222.0.61 () {56 vars in 1186 bytes} [Fri Aug 26 12:00:53 2022] GET /identity/v3/projects/16904d4c978c4c12abd64d625faa3eda => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:53.643250 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d09afaf-9e48-4646-9994-ff6f12190cbe None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:53.644750 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d09afaf-9e48-4646-9994-ff6f12190cbe None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:53.645038 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d09afaf-9e48-4646-9994-ff6f12190cbe None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:53.646279 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 134/270] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:00:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:00:53.652188 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d09afaf-9e48-4646-9994-ff6f12190cbe None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:53.656218 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-47c1b427-e64c-4274-acb9-6774a405ec47 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:00:53.657429 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-47c1b427-e64c-4274-acb9-6774a405ec47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:00:53.703456 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:53.741800 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 137/271] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:00:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:00:53.743155 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e3668-b49f-4c41-ba27-b3030f29fd6f None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:53.743506 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e3668-b49f-4c41-ba27-b3030f29fd6f None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:53.743755 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e3668-b49f-4c41-ba27-b3030f29fd6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:53.763655 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-932e3668-b49f-4c41-ba27-b3030f29fd6f None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:00:53.764899 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-932e3668-b49f-4c41-ba27-b3030f29fd6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:00:53.824841 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d09afaf-9e48-4646-9994-ff6f12190cbe None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:53.838119 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 138/272] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:00:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:00:53.878002 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3769c65-7ea4-4304-bf1d-78c8af3817fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:53.879027 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3769c65-7ea4-4304-bf1d-78c8af3817fa None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:53.879584 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3769c65-7ea4-4304-bf1d-78c8af3817fa None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:53.880057 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3769c65-7ea4-4304-bf1d-78c8af3817fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:53.881039 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3769c65-7ea4-4304-bf1d-78c8af3817fa None admin] RBAC: Adding query filter params (role.id=a60f381814fe4c1da324d13c8e7d5d20, scope.project.id=16904d4c978c4c12abd64d625faa3eda, user.id=8f9b8686397a47409e5d4b2d855191a2) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:00:53.881637 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3769c65-7ea4-4304-bf1d-78c8af3817fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:53.883608 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3769c65-7ea4-4304-bf1d-78c8af3817fa None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:53.896593 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 135/273] 10.222.0.61 () {56 vars in 1374 bytes} [Fri Aug 26 12:00:53 2022] GET /identity/v3/role_assignments?user.id=8f9b8686397a47409e5d4b2d855191a2&scope.project.id=16904d4c978c4c12abd64d625faa3eda&role.id=a60f381814fe4c1da324d13c8e7d5d20 => generated 608 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:54.124413 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932e3668-b49f-4c41-ba27-b3030f29fd6f None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:54.183819 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3769c65-7ea4-4304-bf1d-78c8af3817fa None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:54.218120 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:54.242608 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932e3668-b49f-4c41-ba27-b3030f29fd6f None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:54.244148 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:54.282853 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71187ee0-0f39-44df-89d1-a31be5d6a2f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:54.283761 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71187ee0-0f39-44df-89d1-a31be5d6a2f3 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:54.284106 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71187ee0-0f39-44df-89d1-a31be5d6a2f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:54.284729 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71187ee0-0f39-44df-89d1-a31be5d6a2f3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:54.285913 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71187ee0-0f39-44df-89d1-a31be5d6a2f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:54.287478 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71187ee0-0f39-44df-89d1-a31be5d6a2f3 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:54.290008 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 139/274] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:00:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:54.295956 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71187ee0-0f39-44df-89d1-a31be5d6a2f3 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:54.305553 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3769c65-7ea4-4304-bf1d-78c8af3817fa None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:54.344231 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb80385-afc6-43cc-b84e-fab2bae7535d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:54.344865 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb80385-afc6-43cc-b84e-fab2bae7535d None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:54.345150 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb80385-afc6-43cc-b84e-fab2bae7535d None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:54.345424 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb80385-afc6-43cc-b84e-fab2bae7535d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:54.361700 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb80385-afc6-43cc-b84e-fab2bae7535d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:00:54.367034 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-6eb80385-afc6-43cc-b84e-fab2bae7535d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 12:00:54.367034 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:00:54.367034 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:00:54.367034 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:00:54.367034 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:00:54.367034 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:00:54.367034 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:00:54.367034 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:00:54.367034 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:00:54.367034 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:00:54.367034 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:00:54.367034 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:00:54.367034 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:00:54.367034 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:00:54.367034 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:00:54.367034 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:00:54.368157 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:00:54.368157 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:00:54.368157 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:00:54.368157 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:00:54.368157 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:00:54.368157 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:00:54.368157 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:00:54.368157 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:00:54.368157 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:00:54.368157 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:00:54.368157 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:00:54.368157 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:00:54.368157 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:00:54.368157 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:00:54.368157 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:00:54.368157 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:00:54.369119 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:00:54.369119 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:00:54.369119 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:00:54.369119 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:00:54.369119 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:00:54.369119 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:00:54.369119 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:00:54.369119 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:00:54.369119 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:00:54.369119 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:00:54.369119 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 12:00:54.369119 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:00:54.371348 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 136/275] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:00:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:00:54.373588 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1265f515-ca85-4240-b65c-d97d6ef7ef19 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:54.373877 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1265f515-ca85-4240-b65c-d97d6ef7ef19 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:54.374291 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1265f515-ca85-4240-b65c-d97d6ef7ef19 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:54.375599 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 137/276] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:00:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:00:54.376349 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1265f515-ca85-4240-b65c-d97d6ef7ef19 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:54.423504 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71187ee0-0f39-44df-89d1-a31be5d6a2f3 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:54.460009 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:54.465224 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b933556-dc7b-4db4-a8d3-464c088348ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:54.465833 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b933556-dc7b-4db4-a8d3-464c088348ed None admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:54.466111 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b933556-dc7b-4db4-a8d3-464c088348ed None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:54.467378 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b933556-dc7b-4db4-a8d3-464c088348ed None admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:54.468016 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b933556-dc7b-4db4-a8d3-464c088348ed None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:54.469911 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b933556-dc7b-4db4-a8d3-464c088348ed None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:54.479590 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1265f515-ca85-4240-b65c-d97d6ef7ef19 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:54.493057 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 140/277] 10.222.0.61 () {60 vars in 1132 bytes} [Fri Aug 26 12:00:54 2022] POST /identity/v3/users => generated 293 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:00:54.499415 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce54be41-ea54-4bce-824d-c4f190cd091e None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:54.499415 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce54be41-ea54-4bce-824d-c4f190cd091e None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:54.499415 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce54be41-ea54-4bce-824d-c4f190cd091e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:54.523744 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-ce54be41-ea54-4bce-824d-c4f190cd091e None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:00:54.525208 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-ce54be41-ea54-4bce-824d-c4f190cd091e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:00:54.536377 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fc9779-cdf5-4dd6-91e6-5955e6e6670a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:54.537279 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc9779-cdf5-4dd6-91e6-5955e6e6670a None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:54.537643 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc9779-cdf5-4dd6-91e6-5955e6e6670a None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:54.537987 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc9779-cdf5-4dd6-91e6-5955e6e6670a None admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:54.538864 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fc9779-cdf5-4dd6-91e6-5955e6e6670a None admin] RBAC: Adding query filter params (name=member) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:00:54.539364 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fc9779-cdf5-4dd6-91e6-5955e6e6670a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:54.540899 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fc9779-cdf5-4dd6-91e6-5955e6e6670a None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:54.550733 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:54.553075 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 138/278] 10.222.0.61 () {56 vars in 1101 bytes} [Fri Aug 26 12:00:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:54.561841 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fc9779-cdf5-4dd6-91e6-5955e6e6670a None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:54.610910 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 141/279] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:00:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:00:54.689557 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fc9779-cdf5-4dd6-91e6-5955e6e6670a None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:54.725267 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b9e9e8-4eb7-41d7-8196-4a45932bf880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:54.725822 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9e9e8-4eb7-41d7-8196-4a45932bf880 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:54.726096 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9e9e8-4eb7-41d7-8196-4a45932bf880 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:54.726367 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9e9e8-4eb7-41d7-8196-4a45932bf880 None admin] PATH_INFO: `/v3/users/ef706e6d37094bd2b5756b0b36a1759d` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:54.745596 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b9e9e8-4eb7-41d7-8196-4a45932bf880 None admin] RBAC: Authorizing `identity:get_user(user_id=ef706e6d37094bd2b5756b0b36a1759d)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:54.746703 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b9e9e8-4eb7-41d7-8196-4a45932bf880 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:54.754544 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 139/280] 10.222.0.61 () {56 vars in 1177 bytes} [Fri Aug 26 12:00:54 2022] GET /identity/v3/users/ef706e6d37094bd2b5756b0b36a1759d => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:54.755747 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce54be41-ea54-4bce-824d-c4f190cd091e None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:54.878017 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce54be41-ea54-4bce-824d-c4f190cd091e None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:54.881012 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b9e9e8-4eb7-41d7-8196-4a45932bf880 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:54.886659 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:54.924924 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176995e2-0024-4dac-a91b-30a51bb35eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:54.925541 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176995e2-0024-4dac-a91b-30a51bb35eed None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:54.925823 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176995e2-0024-4dac-a91b-30a51bb35eed None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:54.926096 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176995e2-0024-4dac-a91b-30a51bb35eed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:54.926835 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176995e2-0024-4dac-a91b-30a51bb35eed None admin] RBAC: Adding query filter params (role.id=b2403690f5ad443aad059f5ee024f61c, user.id=ef706e6d37094bd2b5756b0b36a1759d, scope.system=all) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:00:54.927144 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176995e2-0024-4dac-a91b-30a51bb35eed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:54.928659 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176995e2-0024-4dac-a91b-30a51bb35eed None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:54.934358 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 142/281] 10.222.0.61 () {56 vars in 1308 bytes} [Fri Aug 26 12:00:54 2022] GET /identity/v3/role_assignments?user.id=ef706e6d37094bd2b5756b0b36a1759d&scope.system=all&role.id=b2403690f5ad443aad059f5ee024f61c => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:55.020057 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b9e9e8-4eb7-41d7-8196-4a45932bf880 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:55.059397 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bdc2c2-5c4b-4294-a548-72406d3608ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:55.060931 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bdc2c2-5c4b-4294-a548-72406d3608ed None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:55.061439 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bdc2c2-5c4b-4294-a548-72406d3608ed None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:55.061966 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bdc2c2-5c4b-4294-a548-72406d3608ed None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:55.074561 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bdc2c2-5c4b-4294-a548-72406d3608ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:00:55.080646 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-77bdc2c2-5c4b-4294-a548-72406d3608ed None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 12:00:55.080646 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:00:55.080646 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:00:55.080646 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:00:55.080646 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:00:55.080646 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:00:55.080646 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:00:55.080646 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:00:55.080646 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:00:55.080646 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:00:55.080646 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:00:55.080646 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:00:55.080646 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:00:55.080646 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:00:55.080646 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:00:55.080646 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:00:55.081576 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:00:55.081576 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:00:55.081576 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:00:55.081576 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:00:55.081576 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:00:55.081576 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:00:55.081576 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:00:55.081576 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:00:55.081576 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:00:55.081576 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:00:55.081576 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:00:55.081576 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:00:55.081576 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:00:55.081576 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:00:55.081576 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:00:55.081576 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:00:55.082371 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:00:55.082371 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:00:55.082371 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:00:55.082371 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:00:55.082371 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:00:55.082371 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:00:55.082371 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:00:55.082371 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:00:55.082371 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:00:55.082371 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:00:55.082371 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 12:00:55.082371 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:00:55.084697 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 140/282] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:00:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 12:00:55.091007 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176995e2-0024-4dac-a91b-30a51bb35eed None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:55.119194 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:55.210025 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176995e2-0024-4dac-a91b-30a51bb35eed None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:55.250389 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e947269-c11e-40de-aac9-62006e958de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:55.251388 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e947269-c11e-40de-aac9-62006e958de4 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:55.251874 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e947269-c11e-40de-aac9-62006e958de4 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:55.252368 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e947269-c11e-40de-aac9-62006e958de4 None admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:55.253233 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e947269-c11e-40de-aac9-62006e958de4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:00:55.253757 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e947269-c11e-40de-aac9-62006e958de4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:55.255348 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e947269-c11e-40de-aac9-62006e958de4 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:55.265878 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 143/283] 10.222.0.61 () {56 vars in 1101 bytes} [Fri Aug 26 12:00:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:55.271873 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bdc2c2-5c4b-4294-a548-72406d3608ed None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:55.344037 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:55.390779 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bdc2c2-5c4b-4294-a548-72406d3608ed None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:55.434397 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82953af3-073b-4146-82d3-e51d9cf69895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:55.435040 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82953af3-073b-4146-82d3-e51d9cf69895 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:55.435431 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82953af3-073b-4146-82d3-e51d9cf69895 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:55.436268 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82953af3-073b-4146-82d3-e51d9cf69895 None admin] PATH_INFO: `/v3/users/2007314f4a594b7a8bdf4b3932c9bbab` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:55.439905 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82953af3-073b-4146-82d3-e51d9cf69895 None admin] RBAC: Authorizing `identity:get_user(user_id=2007314f4a594b7a8bdf4b3932c9bbab)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:55.441358 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82953af3-073b-4146-82d3-e51d9cf69895 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:55.442966 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 141/284] 10.222.0.61 () {56 vars in 1177 bytes} [Fri Aug 26 12:00:55 2022] GET /identity/v3/users/2007314f4a594b7a8bdf4b3932c9bbab => generated 297 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:55.448604 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e947269-c11e-40de-aac9-62006e958de4 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:55.561520 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:55.588660 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e947269-c11e-40de-aac9-62006e958de4 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:55.616603 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a185fb5a-3105-438e-9a3b-6455da42e7ba None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:55.617362 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a185fb5a-3105-438e-9a3b-6455da42e7ba None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:55.617838 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a185fb5a-3105-438e-9a3b-6455da42e7ba None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:55.619781 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 142/285] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:00:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:00:55.627174 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b860bce3-39cf-4bec-9162-e170d27bfb65 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:55.627779 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b860bce3-39cf-4bec-9162-e170d27bfb65 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:55.628410 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b860bce3-39cf-4bec-9162-e170d27bfb65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:55.647910 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd2e458-e7d6-4db5-98fe-c6b57564572b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:55.648555 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd2e458-e7d6-4db5-98fe-c6b57564572b None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:55.648830 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd2e458-e7d6-4db5-98fe-c6b57564572b None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:55.649095 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd2e458-e7d6-4db5-98fe-c6b57564572b None admin] PATH_INFO: `/v3/projects/16904d4c978c4c12abd64d625faa3eda` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:55.650263 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd2e458-e7d6-4db5-98fe-c6b57564572b None admin] RBAC: Authorizing `identity:get_project(project_id=16904d4c978c4c12abd64d625faa3eda)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:55.651677 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-b860bce3-39cf-4bec-9162-e170d27bfb65 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:00:55.651801 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd2e458-e7d6-4db5-98fe-c6b57564572b None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:55.653180 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-b860bce3-39cf-4bec-9162-e170d27bfb65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:00:55.659229 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 144/286] 10.222.0.61 () {56 vars in 1186 bytes} [Fri Aug 26 12:00:55 2022] GET /identity/v3/projects/16904d4c978c4c12abd64d625faa3eda => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:55.665029 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd2e458-e7d6-4db5-98fe-c6b57564572b None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:55.741768 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 143/287] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:00:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 12:00:55.837662 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd2e458-e7d6-4db5-98fe-c6b57564572b None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:55.884597 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c86176-b404-4ad6-b979-3dd7b6457422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:55.885272 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c86176-b404-4ad6-b979-3dd7b6457422 None admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:55.885554 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c86176-b404-4ad6-b979-3dd7b6457422 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:55.885834 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c86176-b404-4ad6-b979-3dd7b6457422 None admin] PATH_INFO: `/v3/projects/16904d4c978c4c12abd64d625faa3eda/users/2007314f4a594b7a8bdf4b3932c9bbab/roles/12a2a8379bd241a098646f1c8d881614` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:55.889171 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c86176-b404-4ad6-b979-3dd7b6457422 None admin] RBAC: Authorizing `identity:create_grant(project_id=16904d4c978c4c12abd64d625faa3eda, user_id=2007314f4a594b7a8bdf4b3932c9bbab, role_id=12a2a8379bd241a098646f1c8d881614)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:55.890829 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c86176-b404-4ad6-b979-3dd7b6457422 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:55.916830 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 145/288] 10.222.0.61 () {58 vars in 1439 bytes} [Fri Aug 26 12:00:55 2022] PUT /identity/v3/projects/16904d4c978c4c12abd64d625faa3eda/users/2007314f4a594b7a8bdf4b3932c9bbab/roles/12a2a8379bd241a098646f1c8d881614 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 12:00:56.344369 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b860bce3-39cf-4bec-9162-e170d27bfb65 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:56.419148 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:56.476121 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b860bce3-39cf-4bec-9162-e170d27bfb65 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:56.527939 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5e6fea-392b-45e4-987b-9b0e654f2828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:56.528928 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5e6fea-392b-45e4-987b-9b0e654f2828 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:56.530049 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5e6fea-392b-45e4-987b-9b0e654f2828 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:56.530536 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5e6fea-392b-45e4-987b-9b0e654f2828 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:56.542759 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5e6fea-392b-45e4-987b-9b0e654f2828 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:00:56.548880 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-fa5e6fea-392b-45e4-987b-9b0e654f2828 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 12:00:56.548880 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:00:56.548880 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:00:56.548880 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:00:56.548880 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:00:56.548880 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:00:56.548880 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:00:56.548880 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:00:56.548880 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:00:56.548880 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:00:56.548880 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:00:56.548880 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:00:56.548880 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:00:56.548880 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:00:56.548880 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:00:56.548880 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:00:56.549762 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:00:56.549762 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:00:56.549762 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:00:56.549762 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:00:56.549762 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:00:56.549762 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:00:56.549762 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:00:56.549762 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:00:56.549762 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:00:56.549762 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:00:56.549762 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:00:56.549762 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:00:56.549762 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:00:56.549762 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:00:56.549762 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:00:56.549762 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:00:56.550554 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:00:56.550554 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:00:56.550554 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:00:56.550554 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:00:56.550554 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:00:56.550554 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:00:56.550554 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:00:56.550554 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:00:56.550554 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:00:56.550554 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:00:56.550554 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 12:00:56.550554 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:00:56.552869 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 144/289] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:00:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 12:00:56.558701 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c86176-b404-4ad6-b979-3dd7b6457422 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:56.575627 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:56.690001 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c86176-b404-4ad6-b979-3dd7b6457422 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:56.743945 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31d5311-d64c-4572-a293-de6d21853951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:56.744912 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d5311-d64c-4572-a293-de6d21853951 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:56.745390 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d5311-d64c-4572-a293-de6d21853951 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:56.745881 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31d5311-d64c-4572-a293-de6d21853951 None admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:56.746840 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31d5311-d64c-4572-a293-de6d21853951 None admin] RBAC: Adding query filter params (name=member) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:00:56.747387 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31d5311-d64c-4572-a293-de6d21853951 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:56.755387 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31d5311-d64c-4572-a293-de6d21853951 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:56.762657 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 146/290] 10.222.0.61 () {56 vars in 1101 bytes} [Fri Aug 26 12:00:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:56.768439 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5e6fea-392b-45e4-987b-9b0e654f2828 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:56.906219 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5e6fea-392b-45e4-987b-9b0e654f2828 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:56.929339 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:56.946115 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3382cee-8804-4ed0-9fda-a4555ac1b69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:56.946673 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3382cee-8804-4ed0-9fda-a4555ac1b69d None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:56.946953 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3382cee-8804-4ed0-9fda-a4555ac1b69d None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:56.947256 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3382cee-8804-4ed0-9fda-a4555ac1b69d None admin] PATH_INFO: `/v3/users/cb0932d41804458783f0b2063f06bd69` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:56.948414 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3382cee-8804-4ed0-9fda-a4555ac1b69d None admin] RBAC: Authorizing `identity:get_user(user_id=cb0932d41804458783f0b2063f06bd69)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:56.949863 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3382cee-8804-4ed0-9fda-a4555ac1b69d None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:56.952708 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 145/291] 10.222.0.61 () {56 vars in 1177 bytes} [Fri Aug 26 12:00:56 2022] GET /identity/v3/users/cb0932d41804458783f0b2063f06bd69 => generated 297 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:56.958212 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31d5311-d64c-4572-a293-de6d21853951 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:57.091705 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:57.116204 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31d5311-d64c-4572-a293-de6d21853951 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:57.170531 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d5c0e-ebd5-4bac-b680-934b27cdd8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:57.171228 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d5c0e-ebd5-4bac-b680-934b27cdd8b2 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:57.171518 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d5c0e-ebd5-4bac-b680-934b27cdd8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:57.171790 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d5c0e-ebd5-4bac-b680-934b27cdd8b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:57.184877 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd18409a-76f5-4536-80d8-f242f8511712 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:57.185167 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd18409a-76f5-4536-80d8-f242f8511712 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:57.186121 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd18409a-76f5-4536-80d8-f242f8511712 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:57.191911 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 146/292] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:00:57 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:00:57.197035 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0821b28d-8d17-4db3-a76b-f19454305dd7 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:57.197268 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0821b28d-8d17-4db3-a76b-f19454305dd7 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:57.197543 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0821b28d-8d17-4db3-a76b-f19454305dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:57.245922 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290d5c0e-ebd5-4bac-b680-934b27cdd8b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:57.247546 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290d5c0e-ebd5-4bac-b680-934b27cdd8b2 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:57.252262 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-0821b28d-8d17-4db3-a76b-f19454305dd7 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:00:57.253481 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-0821b28d-8d17-4db3-a76b-f19454305dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:00:57.337980 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:57.360236 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 147/293] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:00:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 12:00:57.365713 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d5c0e-ebd5-4bac-b680-934b27cdd8b2 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:57.448062 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 147/294] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:00:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:00:57.510352 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d5c0e-ebd5-4bac-b680-934b27cdd8b2 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:57.528762 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e2776-8456-45f4-963c-a3dbf10e88f3 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:57.529134 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e2776-8456-45f4-963c-a3dbf10e88f3 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:57.530081 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e2776-8456-45f4-963c-a3dbf10e88f3 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:57.532110 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 148/295] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:00:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 12:00:57.537876 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb7b27b-4cbb-4d17-827f-0684b66690f7 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:57.538285 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb7b27b-4cbb-4d17-827f-0684b66690f7 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:57.538567 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb7b27b-4cbb-4d17-827f-0684b66690f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:57.559687 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-3cb7b27b-4cbb-4d17-827f-0684b66690f7 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:00:57.561282 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-3cb7b27b-4cbb-4d17-827f-0684b66690f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:00:57.566229 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a978d7f6-a390-413c-b209-69f1b320ab10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:57.567293 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a978d7f6-a390-413c-b209-69f1b320ab10 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:57.567765 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a978d7f6-a390-413c-b209-69f1b320ab10 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:57.568251 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a978d7f6-a390-413c-b209-69f1b320ab10 None admin] PATH_INFO: `/v3/projects/16904d4c978c4c12abd64d625faa3eda` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:57.569577 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a978d7f6-a390-413c-b209-69f1b320ab10 None admin] RBAC: Authorizing `identity:get_project(project_id=16904d4c978c4c12abd64d625faa3eda)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:57.571313 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a978d7f6-a390-413c-b209-69f1b320ab10 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:57.573391 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 148/296] 10.222.0.61 () {56 vars in 1186 bytes} [Fri Aug 26 12:00:57 2022] GET /identity/v3/projects/16904d4c978c4c12abd64d625faa3eda => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:57.574681 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80c9821-ccc9-4b05-a3ec-1ddf18f9526a None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:57.575151 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80c9821-ccc9-4b05-a3ec-1ddf18f9526a None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:57.580488 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80c9821-ccc9-4b05-a3ec-1ddf18f9526a None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:57.582017 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 149/297] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:00:57 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:00:57.583521 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80c9821-ccc9-4b05-a3ec-1ddf18f9526a None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:57.632160 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 149/298] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:00:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 12:00:57.634019 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3014260c-d50a-4af7-ad56-5426826a526c None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:57.634520 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3014260c-d50a-4af7-ad56-5426826a526c None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:57.635000 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3014260c-d50a-4af7-ad56-5426826a526c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:57.638258 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:57.663057 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-3014260c-d50a-4af7-ad56-5426826a526c None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:00:57.665569 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-3014260c-d50a-4af7-ad56-5426826a526c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:00:57.748677 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80c9821-ccc9-4b05-a3ec-1ddf18f9526a None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:57.753417 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:57.778859 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 150/299] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:00:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:00:57.793912 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3014260c-d50a-4af7-ad56-5426826a526c None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:57.847921 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c900268b-f990-4d93-bb1a-f1a114623668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:57.848545 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c900268b-f990-4d93-bb1a-f1a114623668 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:57.848831 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c900268b-f990-4d93-bb1a-f1a114623668 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:57.849109 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c900268b-f990-4d93-bb1a-f1a114623668 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:57.849858 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c900268b-f990-4d93-bb1a-f1a114623668 None admin] RBAC: Adding query filter params (role.id=b2403690f5ad443aad059f5ee024f61c, scope.project.id=16904d4c978c4c12abd64d625faa3eda, user.id=cb0932d41804458783f0b2063f06bd69) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:00:57.850158 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c900268b-f990-4d93-bb1a-f1a114623668 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:57.851637 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c900268b-f990-4d93-bb1a-f1a114623668 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:57.858532 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 150/300] 10.222.0.61 () {56 vars in 1374 bytes} [Fri Aug 26 12:00:57 2022] GET /identity/v3/role_assignments?user.id=cb0932d41804458783f0b2063f06bd69&scope.project.id=16904d4c978c4c12abd64d625faa3eda&role.id=b2403690f5ad443aad059f5ee024f61c => generated 608 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:00:57.930617 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3014260c-d50a-4af7-ad56-5426826a526c None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:57.937140 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c900268b-f990-4d93-bb1a-f1a114623668 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:57.974393 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821a984b-30c2-4663-a97e-94957ad7d548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:57.975052 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a984b-30c2-4663-a97e-94957ad7d548 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:57.975370 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a984b-30c2-4663-a97e-94957ad7d548 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:57.975647 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a984b-30c2-4663-a97e-94957ad7d548 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:57.988293 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821a984b-30c2-4663-a97e-94957ad7d548 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:00:57.993223 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-821a984b-30c2-4663-a97e-94957ad7d548 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 12:00:57.993223 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:00:57.993223 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:00:57.993223 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:00:57.993223 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:00:57.993223 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:00:57.993223 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:00:57.993223 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:00:57.993223 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:00:57.993223 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:00:57.993223 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:00:57.993223 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:00:57.993223 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:00:57.993223 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:00:57.993223 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:00:57.993223 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:00:57.994166 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:00:57.994166 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:00:57.994166 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:00:57.994166 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:00:57.994166 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:00:57.994166 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:00:57.994166 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:00:57.994166 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:00:57.994166 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:00:57.994166 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:00:57.994166 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:00:57.994166 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:00:57.994166 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:00:57.994166 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:00:57.994166 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:00:57.994166 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:00:57.994988 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:00:57.994988 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:00:57.994988 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:00:57.994988 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:00:57.994988 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:00:57.994988 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:00:57.994988 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:00:57.994988 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:00:57.994988 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:00:57.994988 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:00:57.994988 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 12:00:57.994988 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:00:57.995622 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 151/301] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:00:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 12:00:58.001588 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821a984b-30c2-4663-a97e-94957ad7d548 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:58.089659 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:58.142487 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c900268b-f990-4d93-bb1a-f1a114623668 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:58.145682 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:58.177435 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821a984b-30c2-4663-a97e-94957ad7d548 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:58.185096 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4945b906-c4ce-4d0c-bc1b-69c7959ec4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:58.186206 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4945b906-c4ce-4d0c-bc1b-69c7959ec4b0 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:58.186713 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4945b906-c4ce-4d0c-bc1b-69c7959ec4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:58.187258 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4945b906-c4ce-4d0c-bc1b-69c7959ec4b0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:58.188702 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4945b906-c4ce-4d0c-bc1b-69c7959ec4b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:58.190440 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4945b906-c4ce-4d0c-bc1b-69c7959ec4b0 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:58.192701 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 151/302] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:00:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:00:58.193863 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4945b906-c4ce-4d0c-bc1b-69c7959ec4b0 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:58.214146 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4c2815-4274-492e-b6a1-4b0a42f929ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:58.214748 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4c2815-4274-492e-b6a1-4b0a42f929ff None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:58.215023 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4c2815-4274-492e-b6a1-4b0a42f929ff None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:58.215340 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4c2815-4274-492e-b6a1-4b0a42f929ff None admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:58.215991 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4c2815-4274-492e-b6a1-4b0a42f929ff None admin] RBAC: Adding query filter params (name=reader) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:00:58.216286 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4c2815-4274-492e-b6a1-4b0a42f929ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:58.218489 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4c2815-4274-492e-b6a1-4b0a42f929ff None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:58.226737 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 152/303] 10.222.0.61 () {56 vars in 1101 bytes} [Fri Aug 26 12:00:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:58.227720 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4c2815-4274-492e-b6a1-4b0a42f929ff None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:58.309941 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4945b906-c4ce-4d0c-bc1b-69c7959ec4b0 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:58.335306 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:58.338823 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4c2815-4274-492e-b6a1-4b0a42f929ff None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:58.348310 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630077f3-a297-483d-ad26-46a7f5d5d47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:58.348947 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630077f3-a297-483d-ad26-46a7f5d5d47e None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:58.349228 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630077f3-a297-483d-ad26-46a7f5d5d47e None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:58.349698 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630077f3-a297-483d-ad26-46a7f5d5d47e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:58.360779 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630077f3-a297-483d-ad26-46a7f5d5d47e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:00:58.365805 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-630077f3-a297-483d-ad26-46a7f5d5d47e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 12:00:58.365805 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:00:58.365805 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:00:58.365805 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:00:58.365805 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:00:58.365805 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:00:58.365805 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:00:58.365805 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:00:58.365805 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:00:58.365805 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:00:58.365805 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:00:58.365805 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:00:58.365805 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:00:58.365805 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:00:58.365805 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:00:58.365805 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:00:58.366701 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:00:58.366701 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:00:58.366701 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:00:58.366701 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:00:58.366701 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:00:58.366701 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:00:58.366701 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:00:58.366701 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:00:58.366701 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:00:58.366701 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:00:58.366701 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:00:58.366701 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:00:58.366701 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:00:58.366701 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:00:58.366701 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:00:58.366701 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:00:58.367615 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:00:58.367615 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:00:58.367615 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:00:58.367615 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:00:58.367615 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:00:58.367615 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:00:58.367615 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:00:58.367615 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:00:58.367615 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:00:58.367615 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:00:58.367615 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 12:00:58.367615 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:00:58.367615 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 152/304] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:00:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:00:58.372299 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630077f3-a297-483d-ad26-46a7f5d5d47e None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:58.377283 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8294295b-83ba-42e9-b5d8-f1e1c47e6c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:58.377889 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8294295b-83ba-42e9-b5d8-f1e1c47e6c01 None admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:58.378167 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8294295b-83ba-42e9-b5d8-f1e1c47e6c01 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:58.378437 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8294295b-83ba-42e9-b5d8-f1e1c47e6c01 None admin] PATH_INFO: `/v3/groups` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:58.379054 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8294295b-83ba-42e9-b5d8-f1e1c47e6c01 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:58.380522 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8294295b-83ba-42e9-b5d8-f1e1c47e6c01 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:58.398461 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 153/305] 10.222.0.61 () {60 vars in 1134 bytes} [Fri Aug 26 12:00:58 2022] POST /identity/v3/groups => generated 229 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:00:58.399572 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8294295b-83ba-42e9-b5d8-f1e1c47e6c01 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:58.414292 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:58.515223 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630077f3-a297-483d-ad26-46a7f5d5d47e None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:58.522903 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8294295b-83ba-42e9-b5d8-f1e1c47e6c01 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:58.555575 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41982b3-7a35-4296-93df-0c1cf69bf484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:58.556217 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41982b3-7a35-4296-93df-0c1cf69bf484 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:58.556499 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41982b3-7a35-4296-93df-0c1cf69bf484 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:58.556790 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41982b3-7a35-4296-93df-0c1cf69bf484 None admin] PATH_INFO: `/v3/users/f73889de61064802be83d1334ac128bc` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:58.561749 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa4b1f-f48f-4d09-8fda-0b0068827a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:58.563013 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa4b1f-f48f-4d09-8fda-0b0068827a4f None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:58.563013 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa4b1f-f48f-4d09-8fda-0b0068827a4f None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:58.563013 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa4b1f-f48f-4d09-8fda-0b0068827a4f None admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:58.563539 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfa4b1f-f48f-4d09-8fda-0b0068827a4f None admin] RBAC: Adding query filter params (name=member) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:00:58.563884 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfa4b1f-f48f-4d09-8fda-0b0068827a4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:58.565198 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfa4b1f-f48f-4d09-8fda-0b0068827a4f None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:58.571333 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41982b3-7a35-4296-93df-0c1cf69bf484 None admin] RBAC: Authorizing `identity:get_user(user_id=f73889de61064802be83d1334ac128bc)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:58.572790 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41982b3-7a35-4296-93df-0c1cf69bf484 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:58.573843 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 154/306] 10.222.0.61 () {56 vars in 1101 bytes} [Fri Aug 26 12:00:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:58.574443 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 153/307] 10.222.0.61 () {56 vars in 1177 bytes} [Fri Aug 26 12:00:58 2022] GET /identity/v3/users/f73889de61064802be83d1334ac128bc => generated 293 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:58.575563 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af53fd8c-b700-48d7-b0cc-5c1ce3216350 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:58.575921 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af53fd8c-b700-48d7-b0cc-5c1ce3216350 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:58.576192 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af53fd8c-b700-48d7-b0cc-5c1ce3216350 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:58.579385 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 155/308] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:00:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:00:58.580127 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41982b3-7a35-4296-93df-0c1cf69bf484 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:58.580559 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af53fd8c-b700-48d7-b0cc-5c1ce3216350 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:58.609317 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:58.684639 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:58.693050 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af53fd8c-b700-48d7-b0cc-5c1ce3216350 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:58.695377 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41982b3-7a35-4296-93df-0c1cf69bf484 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:58.729800 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2817795f-6075-45ff-b9e7-b1f9e6966100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:58.730408 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2817795f-6075-45ff-b9e7-b1f9e6966100 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:58.730735 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2817795f-6075-45ff-b9e7-b1f9e6966100 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:58.730968 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2817795f-6075-45ff-b9e7-b1f9e6966100 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:58.731567 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3acc8f-f6cf-4f4e-b39d-47fd575c31eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:58.731751 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2817795f-6075-45ff-b9e7-b1f9e6966100 None admin] RBAC: Adding query filter params (role.id=12a2a8379bd241a098646f1c8d881614, user.id=f73889de61064802be83d1334ac128bc, scope.system=all) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:00:58.732174 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2817795f-6075-45ff-b9e7-b1f9e6966100 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:58.732930 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3acc8f-f6cf-4f4e-b39d-47fd575c31eb None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:58.733354 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3acc8f-f6cf-4f4e-b39d-47fd575c31eb None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:58.733693 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3acc8f-f6cf-4f4e-b39d-47fd575c31eb None admin] PATH_INFO: `/v3/users/ef706e6d37094bd2b5756b0b36a1759d` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:58.734467 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2817795f-6075-45ff-b9e7-b1f9e6966100 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:58.734946 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3acc8f-f6cf-4f4e-b39d-47fd575c31eb None admin] RBAC: Authorizing `identity:get_user(user_id=ef706e6d37094bd2b5756b0b36a1759d)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:58.736476 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3acc8f-f6cf-4f4e-b39d-47fd575c31eb None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:58.738451 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 154/309] 10.222.0.61 () {56 vars in 1177 bytes} [Fri Aug 26 12:00:58 2022] GET /identity/v3/users/ef706e6d37094bd2b5756b0b36a1759d => generated 293 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:00:58.742736 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5ace46-179f-49a8-a74e-258ac9cc6f9d None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:58.742736 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5ace46-179f-49a8-a74e-258ac9cc6f9d None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:58.742736 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5ace46-179f-49a8-a74e-258ac9cc6f9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:58.744790 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 156/310] 10.222.0.61 () {56 vars in 1308 bytes} [Fri Aug 26 12:00:58 2022] GET /identity/v3/role_assignments?user.id=f73889de61064802be83d1334ac128bc&scope.system=all&role.id=12a2a8379bd241a098646f1c8d881614 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:00:58.745409 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2817795f-6075-45ff-b9e7-b1f9e6966100 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:58.761583 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-5e5ace46-179f-49a8-a74e-258ac9cc6f9d None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:00:58.762901 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-5e5ace46-179f-49a8-a74e-258ac9cc6f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:00:58.863475 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:58.867226 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 155/311] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:00:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:00:58.886090 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2817795f-6075-45ff-b9e7-b1f9e6966100 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:58.924571 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a8d7aa-cd85-47ab-b1fd-56909d6259f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:58.925487 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a8d7aa-cd85-47ab-b1fd-56909d6259f3 None admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:58.925906 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a8d7aa-cd85-47ab-b1fd-56909d6259f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:58.926319 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a8d7aa-cd85-47ab-b1fd-56909d6259f3 None admin] PATH_INFO: `/v3/system/users/ef706e6d37094bd2b5756b0b36a1759d/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:58.928044 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a8d7aa-cd85-47ab-b1fd-56909d6259f3 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=ef706e6d37094bd2b5756b0b36a1759d, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:58.929559 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a8d7aa-cd85-47ab-b1fd-56909d6259f3 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:58.941369 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 157/312] 10.222.0.61 () {58 vars in 1334 bytes} [Fri Aug 26 12:00:58 2022] PUT /identity/v3/system/users/ef706e6d37094bd2b5756b0b36a1759d/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:00:59.155940 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5ace46-179f-49a8-a74e-258ac9cc6f9d None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:59.247735 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:59.262885 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5ace46-179f-49a8-a74e-258ac9cc6f9d None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:59.307759 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d48aa0c-b338-41ad-b3a8-ce4e96bfa651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:59.308389 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48aa0c-b338-41ad-b3a8-ce4e96bfa651 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:59.308662 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48aa0c-b338-41ad-b3a8-ce4e96bfa651 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:59.308929 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48aa0c-b338-41ad-b3a8-ce4e96bfa651 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:59.319600 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d48aa0c-b338-41ad-b3a8-ce4e96bfa651 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:00:59.324385 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-1d48aa0c-b338-41ad-b3a8-ce4e96bfa651 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 12:00:59.324385 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:00:59.324385 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:00:59.324385 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:00:59.324385 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:00:59.324385 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:00:59.324385 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:00:59.324385 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:00:59.324385 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:00:59.324385 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:00:59.324385 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:00:59.324385 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:00:59.324385 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:00:59.324385 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:00:59.324385 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:00:59.324385 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:00:59.325261 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:00:59.325261 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:00:59.325261 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:00:59.325261 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:00:59.325261 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:00:59.325261 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:00:59.325261 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:00:59.325261 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:00:59.325261 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:00:59.325261 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:00:59.325261 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:00:59.325261 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:00:59.325261 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:00:59.325261 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:00:59.325261 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:00:59.325261 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:00:59.326130 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:00:59.326130 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:00:59.326130 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:00:59.326130 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:00:59.326130 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:00:59.326130 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:00:59.326130 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:00:59.326130 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:00:59.326130 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:00:59.326130 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:00:59.326130 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 12:00:59.326130 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:00:59.330397 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 156/313] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:00:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:00:59.331802 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a8d7aa-cd85-47ab-b1fd-56909d6259f3 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:59.446446 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a8d7aa-cd85-47ab-b1fd-56909d6259f3 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:59.481404 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472ccf7f-c00d-4a3f-8d36-14f5db051a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:59.483007 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ccf7f-c00d-4a3f-8d36-14f5db051a53 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:59.483522 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ccf7f-c00d-4a3f-8d36-14f5db051a53 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:59.483993 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ccf7f-c00d-4a3f-8d36-14f5db051a53 None admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:59.484843 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472ccf7f-c00d-4a3f-8d36-14f5db051a53 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:00:59.485338 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472ccf7f-c00d-4a3f-8d36-14f5db051a53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:59.486928 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472ccf7f-c00d-4a3f-8d36-14f5db051a53 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:59.495493 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 158/314] 10.222.0.61 () {56 vars in 1101 bytes} [Fri Aug 26 12:00:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:59.507238 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d48aa0c-b338-41ad-b3a8-ce4e96bfa651 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:59.610129 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d48aa0c-b338-41ad-b3a8-ce4e96bfa651 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:59.647277 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eec24aa-8f7c-41d6-96b6-4701445bf05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:59.647918 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eec24aa-8f7c-41d6-96b6-4701445bf05a None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:59.648198 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eec24aa-8f7c-41d6-96b6-4701445bf05a None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:59.648471 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eec24aa-8f7c-41d6-96b6-4701445bf05a None admin] PATH_INFO: `/v3/users/2007314f4a594b7a8bdf4b3932c9bbab` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:59.649609 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eec24aa-8f7c-41d6-96b6-4701445bf05a None admin] RBAC: Authorizing `identity:get_user(user_id=2007314f4a594b7a8bdf4b3932c9bbab)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:59.651051 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eec24aa-8f7c-41d6-96b6-4701445bf05a None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:59.652673 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 157/315] 10.222.0.61 () {56 vars in 1177 bytes} [Fri Aug 26 12:00:59 2022] GET /identity/v3/users/2007314f4a594b7a8bdf4b3932c9bbab => generated 297 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:00:59.659367 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472ccf7f-c00d-4a3f-8d36-14f5db051a53 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:00:59.675470 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:59.769283 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472ccf7f-c00d-4a3f-8d36-14f5db051a53 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:00:59.805917 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbe1d2f-e6df-4237-bb20-c252f235a40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:00:59.806530 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe1d2f-e6df-4237-bb20-c252f235a40f None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:00:59.806806 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe1d2f-e6df-4237-bb20-c252f235a40f None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:00:59.807079 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe1d2f-e6df-4237-bb20-c252f235a40f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:00:59.883892 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbe1d2f-e6df-4237-bb20-c252f235a40f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:00:59.885365 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbe1d2f-e6df-4237-bb20-c252f235a40f None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:00:59.899145 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:00:59.988567 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 159/316] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:00:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 12:00:59.993844 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eec24aa-8f7c-41d6-96b6-4701445bf05a None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:00.014798 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:00.102599 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eec24aa-8f7c-41d6-96b6-4701445bf05a None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:00.142785 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497bc0f5-081f-4af1-ac24-e1f64a2fe34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:00.143955 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497bc0f5-081f-4af1-ac24-e1f64a2fe34e None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:00.144457 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497bc0f5-081f-4af1-ac24-e1f64a2fe34e None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:00.144923 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497bc0f5-081f-4af1-ac24-e1f64a2fe34e None admin] PATH_INFO: `/v3/projects/16904d4c978c4c12abd64d625faa3eda` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:00.146249 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497bc0f5-081f-4af1-ac24-e1f64a2fe34e None admin] RBAC: Authorizing `identity:get_project(project_id=16904d4c978c4c12abd64d625faa3eda)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:00.147947 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497bc0f5-081f-4af1-ac24-e1f64a2fe34e None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:00.150080 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 158/317] 10.222.0.61 () {56 vars in 1186 bytes} [Fri Aug 26 12:00:59 2022] GET /identity/v3/projects/16904d4c978c4c12abd64d625faa3eda => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:00.156623 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbe1d2f-e6df-4237-bb20-c252f235a40f None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:00.259419 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbe1d2f-e6df-4237-bb20-c252f235a40f None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:00.273537 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:00.294584 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fc839e-84b0-4902-a0f3-53780f338e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:00.295285 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc839e-84b0-4902-a0f3-53780f338e17 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:00.295811 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc839e-84b0-4902-a0f3-53780f338e17 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:00.296084 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fc839e-84b0-4902-a0f3-53780f338e17 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:00.296808 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fc839e-84b0-4902-a0f3-53780f338e17 None admin] RBAC: Adding query filter params (role.id=12a2a8379bd241a098646f1c8d881614, scope.project.id=16904d4c978c4c12abd64d625faa3eda, user.id=2007314f4a594b7a8bdf4b3932c9bbab) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:00.297116 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fc839e-84b0-4902-a0f3-53780f338e17 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:00.298554 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fc839e-84b0-4902-a0f3-53780f338e17 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:00.304827 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 160/318] 10.222.0.61 () {56 vars in 1374 bytes} [Fri Aug 26 12:01:00 2022] GET /identity/v3/role_assignments?user.id=2007314f4a594b7a8bdf4b3932c9bbab&scope.project.id=16904d4c978c4c12abd64d625faa3eda&role.id=12a2a8379bd241a098646f1c8d881614 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:00.382054 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3285f-fdf7-4665-a509-b97beb0c563e None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:00.382897 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3285f-fdf7-4665-a509-b97beb0c563e None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:00.383761 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3285f-fdf7-4665-a509-b97beb0c563e None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:00.385718 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 159/319] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:01:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:01:00.394100 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c42b8d-4c72-4ed3-9a5a-dbd5f44a1b1b None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:00.394701 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c42b8d-4c72-4ed3-9a5a-dbd5f44a1b1b None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:00.395179 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c42b8d-4c72-4ed3-9a5a-dbd5f44a1b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:00.415733 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-f3c42b8d-4c72-4ed3-9a5a-dbd5f44a1b1b None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:01:00.417516 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-f3c42b8d-4c72-4ed3-9a5a-dbd5f44a1b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:01:00.494733 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 161/320] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:01:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:01:00.764529 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d3285f-fdf7-4665-a509-b97beb0c563e None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:00.802961 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec4dd1-81b3-4e7c-ae3c-0889b0c9ee06 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:00.803524 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec4dd1-81b3-4e7c-ae3c-0889b0c9ee06 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:00.803524 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec4dd1-81b3-4e7c-ae3c-0889b0c9ee06 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:00.804843 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 162/321] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:01:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:01:00.809809 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc0a46-2521-4b51-b0ba-c9824bbe6cf0 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:00.810094 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc0a46-2521-4b51-b0ba-c9824bbe6cf0 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:00.810420 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc0a46-2521-4b51-b0ba-c9824bbe6cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:00.828214 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-77cc0a46-2521-4b51-b0ba-c9824bbe6cf0 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:01:00.829376 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-77cc0a46-2521-4b51-b0ba-c9824bbe6cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:01:00.859300 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:00.877641 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d3285f-fdf7-4665-a509-b97beb0c563e None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:00.903493 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 163/322] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:01:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 12:01:00.917271 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d50fe36-aa47-4110-851a-91e8fb347ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:00.918008 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d50fe36-aa47-4110-851a-91e8fb347ec3 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:00.918402 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d50fe36-aa47-4110-851a-91e8fb347ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:00.918745 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d50fe36-aa47-4110-851a-91e8fb347ec3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:00.920761 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d50fe36-aa47-4110-851a-91e8fb347ec3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:00.922321 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d50fe36-aa47-4110-851a-91e8fb347ec3 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:00.927226 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 160/323] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:01:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:01:00.929493 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cc0a46-2521-4b51-b0ba-c9824bbe6cf0 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:00.964023 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:01.014742 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c33ff38-8c21-4886-b5ce-4760bde65579 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:01.015276 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c33ff38-8c21-4886-b5ce-4760bde65579 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:01.015704 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c33ff38-8c21-4886-b5ce-4760bde65579 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:01.017044 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 161/324] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:01:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 12:01:01.021811 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af38432-8491-4497-af11-6d2075350725 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:01.022094 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af38432-8491-4497-af11-6d2075350725 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:01.022371 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af38432-8491-4497-af11-6d2075350725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:01.031920 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cc0a46-2521-4b51-b0ba-c9824bbe6cf0 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:01.040961 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-4af38432-8491-4497-af11-6d2075350725 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:01:01.042147 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-4af38432-8491-4497-af11-6d2075350725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:01:01.069520 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1973cc0-126f-4fe5-b67f-ad3efab3a1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:01.070125 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1973cc0-126f-4fe5-b67f-ad3efab3a1f6 None admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:01.070399 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1973cc0-126f-4fe5-b67f-ad3efab3a1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:01.070667 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1973cc0-126f-4fe5-b67f-ad3efab3a1f6 None admin] PATH_INFO: `/v3/groups` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:01.071318 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1973cc0-126f-4fe5-b67f-ad3efab3a1f6 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:01.072742 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1973cc0-126f-4fe5-b67f-ad3efab3a1f6 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:01.089280 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 164/325] 10.222.0.61 () {60 vars in 1134 bytes} [Fri Aug 26 12:01:00 2022] POST /identity/v3/groups => generated 226 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:01:01.118760 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 162/326] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:01:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 12:01:01.181687 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1973cc0-126f-4fe5-b67f-ad3efab3a1f6 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:01.282704 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:01.289215 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1973cc0-126f-4fe5-b67f-ad3efab3a1f6 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:01.333831 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a4e29-5f5f-49f7-8f49-54514604c883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:01.334865 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a4e29-5f5f-49f7-8f49-54514604c883 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:01.335146 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a4e29-5f5f-49f7-8f49-54514604c883 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:01.335539 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a4e29-5f5f-49f7-8f49-54514604c883 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:01.348401 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488a4e29-5f5f-49f7-8f49-54514604c883 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:01:01.354154 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-488a4e29-5f5f-49f7-8f49-54514604c883 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 12:01:01.354154 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:01.354154 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:01.354154 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:01.354154 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:01.354154 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:01.354154 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:01.354154 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:01.354154 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:01.354154 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:01.354154 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:01.354154 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:01.354154 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:01:01.354154 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:01:01.354154 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:01:01.354154 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:01:01.355081 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:01:01.355081 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:01:01.355081 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:01:01.355081 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:01:01.355081 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:01.355081 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:01.355081 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:01.355081 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:01.355081 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:01.355081 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:01.355081 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:01.355081 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:01.355081 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:01.355081 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:01.355081 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:01.355081 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:01.355929 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:01.355929 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:01.355929 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:01.355929 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:01.355929 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:01:01.355929 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:01:01.355929 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:01:01.355929 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:01:01.355929 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:01:01.355929 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:01:01.355929 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 12:01:01.355929 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:01.356551 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 165/327] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:01:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:01.371599 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af38432-8491-4497-af11-6d2075350725 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:01.390155 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a4e29-5f5f-49f7-8f49-54514604c883 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:01.486449 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:01.515352 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af38432-8491-4497-af11-6d2075350725 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:01.525812 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a4e29-5f5f-49f7-8f49-54514604c883 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:01.559484 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dd36e6-dc5c-4db4-b081-2f081bb2d869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:01.560364 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd36e6-dc5c-4db4-b081-2f081bb2d869 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:01.560638 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd36e6-dc5c-4db4-b081-2f081bb2d869 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:01.560907 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd36e6-dc5c-4db4-b081-2f081bb2d869 None admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:01.562223 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dd36e6-dc5c-4db4-b081-2f081bb2d869 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:01.562538 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dd36e6-dc5c-4db4-b081-2f081bb2d869 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:01.564237 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dd36e6-dc5c-4db4-b081-2f081bb2d869 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:01.567111 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cefae3b-a005-4da2-9ce2-c7ea794a1aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:01.567842 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cefae3b-a005-4da2-9ce2-c7ea794a1aef None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:01.568183 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cefae3b-a005-4da2-9ce2-c7ea794a1aef None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:01.568519 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cefae3b-a005-4da2-9ce2-c7ea794a1aef None admin] PATH_INFO: `/v3/roles/member` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:01.575281 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 163/328] 10.222.0.61 () {56 vars in 1101 bytes} [Fri Aug 26 12:01:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:01.581299 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dd36e6-dc5c-4db4-b081-2f081bb2d869 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:01.589849 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cefae3b-a005-4da2-9ce2-c7ea794a1aef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:01:01.595186 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-7cefae3b-a005-4da2-9ce2-c7ea794a1aef None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 12:01:01.595186 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:01.595186 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:01.595186 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:01.595186 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:01.595186 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:01.595186 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:01.595186 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:01.595186 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:01.595186 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:01.595186 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:01.595186 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:01.595186 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:01:01.595186 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:01:01.595186 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:01:01.595186 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:01:01.596116 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:01:01.596116 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:01:01.596116 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:01:01.596116 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:01:01.596116 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:01.596116 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:01.596116 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:01.596116 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:01.596116 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:01.596116 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:01.596116 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:01.596116 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:01.596116 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:01.596116 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:01.596116 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:01.596116 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:01.596924 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:01.596924 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:01.596924 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:01.596924 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:01.596924 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:01:01.596924 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:01:01.596924 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:01:01.596924 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:01:01.596924 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:01:01.596924 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:01:01.596924 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 12:01:01.596924 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:01.597539 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 166/329] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:01:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 12:01:01.602970 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cefae3b-a005-4da2-9ce2-c7ea794a1aef None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:01.625760 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:01.705078 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dd36e6-dc5c-4db4-b081-2f081bb2d869 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:01.738309 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cefae3b-a005-4da2-9ce2-c7ea794a1aef None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:01.750866 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3dc7fa-a81f-484e-8702-6a7d15307088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:01.752029 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3dc7fa-a81f-484e-8702-6a7d15307088 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:01.752551 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3dc7fa-a81f-484e-8702-6a7d15307088 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:01.753010 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3dc7fa-a81f-484e-8702-6a7d15307088 None admin] PATH_INFO: `/v3/users/f73889de61064802be83d1334ac128bc` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:01.754465 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3dc7fa-a81f-484e-8702-6a7d15307088 None admin] RBAC: Authorizing `identity:get_user(user_id=f73889de61064802be83d1334ac128bc)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:01.756295 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3dc7fa-a81f-484e-8702-6a7d15307088 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:01.764087 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 164/330] 10.222.0.61 () {56 vars in 1177 bytes} [Fri Aug 26 12:01:01 2022] GET /identity/v3/users/f73889de61064802be83d1334ac128bc => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:01:01.767914 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3dc7fa-a81f-484e-8702-6a7d15307088 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:01.781060 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a94a5f6-c38e-4a98-a28f-a36852e2a0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:01.781679 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a94a5f6-c38e-4a98-a28f-a36852e2a0cf None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:01.781959 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a94a5f6-c38e-4a98-a28f-a36852e2a0cf None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:01.782232 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a94a5f6-c38e-4a98-a28f-a36852e2a0cf None admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:01.785825 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a94a5f6-c38e-4a98-a28f-a36852e2a0cf None admin] RBAC: Adding query filter params (name=member) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:01.785825 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a94a5f6-c38e-4a98-a28f-a36852e2a0cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:01.785825 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a94a5f6-c38e-4a98-a28f-a36852e2a0cf None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:01.790996 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:01.798267 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 167/331] 10.222.0.61 () {56 vars in 1101 bytes} [Fri Aug 26 12:01:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:01:01.804083 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a94a5f6-c38e-4a98-a28f-a36852e2a0cf None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:01.899815 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3dc7fa-a81f-484e-8702-6a7d15307088 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:01.914440 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:01.939137 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a94a5f6-c38e-4a98-a28f-a36852e2a0cf None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:01.944824 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe811b-3317-40d9-b73f-978c2934cc1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:01.945500 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe811b-3317-40d9-b73f-978c2934cc1e None admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:01.945781 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe811b-3317-40d9-b73f-978c2934cc1e None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:01.946055 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe811b-3317-40d9-b73f-978c2934cc1e None admin] PATH_INFO: `/v3/system/users/f73889de61064802be83d1334ac128bc/roles/12a2a8379bd241a098646f1c8d881614` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:01.950183 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fe811b-3317-40d9-b73f-978c2934cc1e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=f73889de61064802be83d1334ac128bc, role_id=12a2a8379bd241a098646f1c8d881614)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:01.952055 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fe811b-3317-40d9-b73f-978c2934cc1e None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:01.969639 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 165/332] 10.222.0.61 () {58 vars in 1334 bytes} [Fri Aug 26 12:01:01 2022] PUT /identity/v3/system/users/f73889de61064802be83d1334ac128bc/roles/12a2a8379bd241a098646f1c8d881614 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:01:01.980851 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3ffab6-b560-4b66-911c-64af76a61f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:01.981507 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ffab6-b560-4b66-911c-64af76a61f47 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:01.981831 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ffab6-b560-4b66-911c-64af76a61f47 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:01.982643 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ffab6-b560-4b66-911c-64af76a61f47 None admin] PATH_INFO: `/v3/users/ef706e6d37094bd2b5756b0b36a1759d` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:01.984042 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3ffab6-b560-4b66-911c-64af76a61f47 None admin] RBAC: Authorizing `identity:get_user(user_id=ef706e6d37094bd2b5756b0b36a1759d)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:01.985656 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3ffab6-b560-4b66-911c-64af76a61f47 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:01.987701 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 168/333] 10.222.0.61 () {56 vars in 1177 bytes} [Fri Aug 26 12:01:01 2022] GET /identity/v3/users/ef706e6d37094bd2b5756b0b36a1759d => generated 293 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:01.994139 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe811b-3317-40d9-b73f-978c2934cc1e None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:02.096287 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:02.125233 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe811b-3317-40d9-b73f-978c2934cc1e None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:02.161947 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadbf14c-2b3b-4548-ad07-8c68a731a1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:02.162487 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadbf14c-2b3b-4548-ad07-8c68a731a1e1 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:02.162814 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadbf14c-2b3b-4548-ad07-8c68a731a1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:02.163093 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadbf14c-2b3b-4548-ad07-8c68a731a1e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:02.163848 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadbf14c-2b3b-4548-ad07-8c68a731a1e1 None admin] RBAC: Adding query filter params (role.id=b2403690f5ad443aad059f5ee024f61c, user.id=ef706e6d37094bd2b5756b0b36a1759d, scope.system=all) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:02.164152 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadbf14c-2b3b-4548-ad07-8c68a731a1e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:02.165606 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadbf14c-2b3b-4548-ad07-8c68a731a1e1 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:02.179787 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 166/334] 10.222.0.61 () {56 vars in 1308 bytes} [Fri Aug 26 12:01:01 2022] GET /identity/v3/role_assignments?user.id=ef706e6d37094bd2b5756b0b36a1759d&scope.system=all&role.id=b2403690f5ad443aad059f5ee024f61c => generated 510 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:03.692283 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd606ce-ad62-45e7-bc91-6d8c4947fbae None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:03.692864 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd606ce-ad62-45e7-bc91-6d8c4947fbae None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:03.692864 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd606ce-ad62-45e7-bc91-6d8c4947fbae None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:03.694410 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 169/335] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:01:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:01:03.699456 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee709ec7-37ea-42dd-b877-e004061aeff3 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:03.700253 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee709ec7-37ea-42dd-b877-e004061aeff3 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:03.701303 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee709ec7-37ea-42dd-b877-e004061aeff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:03.724186 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-ee709ec7-37ea-42dd-b877-e004061aeff3 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:01:03.725692 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-ee709ec7-37ea-42dd-b877-e004061aeff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:01:03.751589 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a0df6b-ffb7-46a5-90d0-f38bb25eb91e None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:03.752130 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a0df6b-ffb7-46a5-90d0-f38bb25eb91e None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:03.752601 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a0df6b-ffb7-46a5-90d0-f38bb25eb91e None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:03.754375 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 170/336] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:01:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:01:03.759970 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf609d0-dce9-46ce-8d11-7e1ff3ec222b None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:03.760491 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf609d0-dce9-46ce-8d11-7e1ff3ec222b None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:03.760978 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf609d0-dce9-46ce-8d11-7e1ff3ec222b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:03.790405 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-3bf609d0-dce9-46ce-8d11-7e1ff3ec222b None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:01:03.792924 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-3bf609d0-dce9-46ce-8d11-7e1ff3ec222b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:01:03.899410 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 167/337] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:01:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:01:03.935892 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 171/338] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:01:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:01:04.282312 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee709ec7-37ea-42dd-b877-e004061aeff3 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:04.352784 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:04.395517 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee709ec7-37ea-42dd-b877-e004061aeff3 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:04.438476 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d714c75c-d9b8-4b3c-a038-977b97aefc9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:04.439481 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d714c75c-d9b8-4b3c-a038-977b97aefc9c None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:04.439958 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d714c75c-d9b8-4b3c-a038-977b97aefc9c None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:04.440459 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d714c75c-d9b8-4b3c-a038-977b97aefc9c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:04.457368 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d714c75c-d9b8-4b3c-a038-977b97aefc9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:01:04.462458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-d714c75c-d9b8-4b3c-a038-977b97aefc9c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 12:01:04.462458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:04.462458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:04.462458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:04.462458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:04.462458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:04.462458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:04.462458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:04.462458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:04.462458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:04.462458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:04.462458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:04.462458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:01:04.462458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:01:04.462458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:01:04.462458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:01:04.463376 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:01:04.463376 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:01:04.463376 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:01:04.463376 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:01:04.463376 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:04.463376 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:04.463376 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:04.463376 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:04.463376 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:04.463376 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:04.463376 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:04.463376 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:04.463376 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:04.463376 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:04.463376 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:04.463376 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:04.464169 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:04.464169 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:04.464169 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:04.464169 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:04.464169 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:01:04.464169 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:01:04.464169 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:01:04.464169 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:01:04.464169 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:01:04.464169 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:01:04.464169 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 12:01:04.464169 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:04.467363 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 168/339] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:01:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:04.472973 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf609d0-dce9-46ce-8d11-7e1ff3ec222b None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:04.483626 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5b35a-6ca9-4c7d-b7ac-2f4f7ec9cc88 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:04.484213 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5b35a-6ca9-4c7d-b7ac-2f4f7ec9cc88 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:04.484685 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5b35a-6ca9-4c7d-b7ac-2f4f7ec9cc88 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:04.486490 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 169/340] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:01:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 12:01:04.487444 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c5b35a-6ca9-4c7d-b7ac-2f4f7ec9cc88 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:04.576252 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:04.606778 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf609d0-dce9-46ce-8d11-7e1ff3ec222b None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:04.651134 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d02d21d-8b4e-4f70-863f-0bcb0d0bbea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:04.652122 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02d21d-8b4e-4f70-863f-0bcb0d0bbea6 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:04.652594 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02d21d-8b4e-4f70-863f-0bcb0d0bbea6 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:04.653081 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02d21d-8b4e-4f70-863f-0bcb0d0bbea6 None admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:04.657266 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d02d21d-8b4e-4f70-863f-0bcb0d0bbea6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:04.657832 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d02d21d-8b4e-4f70-863f-0bcb0d0bbea6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:04.659451 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d02d21d-8b4e-4f70-863f-0bcb0d0bbea6 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:04.672465 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 172/341] 10.222.0.61 () {56 vars in 1101 bytes} [Fri Aug 26 12:01:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:04.673831 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed56fd-7783-47d7-a08a-aec40cb0433d None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:04.674118 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed56fd-7783-47d7-a08a-aec40cb0433d None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:04.674390 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed56fd-7783-47d7-a08a-aec40cb0433d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:04.681721 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c5b35a-6ca9-4c7d-b7ac-2f4f7ec9cc88 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:04.704868 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-d7ed56fd-7783-47d7-a08a-aec40cb0433d None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:01:04.706059 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-d7ed56fd-7783-47d7-a08a-aec40cb0433d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:01:04.724914 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:04.748867 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bb129e-89e8-4209-bf8a-f6ddaebf5cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:04.749817 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bb129e-89e8-4209-bf8a-f6ddaebf5cef None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:04.750112 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bb129e-89e8-4209-bf8a-f6ddaebf5cef None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:04.750383 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bb129e-89e8-4209-bf8a-f6ddaebf5cef None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:04.764058 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bb129e-89e8-4209-bf8a-f6ddaebf5cef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:01:04.769031 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-94bb129e-89e8-4209-bf8a-f6ddaebf5cef None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 12:01:04.769031 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:04.769031 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:04.769031 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:04.769031 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:04.769031 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:04.769031 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:04.769031 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:04.769031 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:04.769031 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:04.769031 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:04.769031 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:04.769031 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:01:04.769031 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:01:04.769031 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:01:04.769031 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:01:04.770136 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:01:04.770136 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:01:04.770136 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:01:04.770136 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:01:04.770136 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:04.770136 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:04.770136 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:04.770136 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:04.770136 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:04.770136 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:04.770136 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:04.770136 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:04.770136 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:04.770136 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:04.770136 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:04.770136 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:04.771104 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:04.771104 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:04.771104 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:04.771104 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:04.771104 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:01:04.771104 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:01:04.771104 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:01:04.771104 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:01:04.771104 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:01:04.771104 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:01:04.771104 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 12:01:04.771104 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:04.773499 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 170/342] 10.222.0.61 () {56 vars in 1114 bytes} [Fri Aug 26 12:01:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:04.774476 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bb129e-89e8-4209-bf8a-f6ddaebf5cef None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:04.865112 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 173/343] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:01:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:01:04.865975 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ed56fd-7783-47d7-a08a-aec40cb0433d None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:04.892272 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bb129e-89e8-4209-bf8a-f6ddaebf5cef None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:04.937214 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d96d3-b741-4aec-8fcb-c759072a50dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:04.937832 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d96d3-b741-4aec-8fcb-c759072a50dc None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:04.938143 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d96d3-b741-4aec-8fcb-c759072a50dc None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:04.938496 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d96d3-b741-4aec-8fcb-c759072a50dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:04.969936 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:05.020012 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:05.023271 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ed56fd-7783-47d7-a08a-aec40cb0433d None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:05.042550 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1d96d3-b741-4aec-8fcb-c759072a50dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:05.044281 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1d96d3-b741-4aec-8fcb-c759072a50dc None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:05.100485 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0f05f8-f8d5-4101-97dd-40b131f4e56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:05.101373 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0f05f8-f8d5-4101-97dd-40b131f4e56e None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:05.101802 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0f05f8-f8d5-4101-97dd-40b131f4e56e None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:05.102210 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0f05f8-f8d5-4101-97dd-40b131f4e56e None admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:05.103030 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0f05f8-f8d5-4101-97dd-40b131f4e56e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:05.103505 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0f05f8-f8d5-4101-97dd-40b131f4e56e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:05.105030 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0f05f8-f8d5-4101-97dd-40b131f4e56e None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:05.121463 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 174/344] 10.222.0.61 () {56 vars in 1111 bytes} [Fri Aug 26 12:01:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:05.127993 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0f05f8-f8d5-4101-97dd-40b131f4e56e None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:05.199528 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 171/345] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:01:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 12:01:05.204489 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aff9d0-8c4c-4ad6-9973-e38a76f67212 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:05.204942 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aff9d0-8c4c-4ad6-9973-e38a76f67212 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:05.205384 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aff9d0-8c4c-4ad6-9973-e38a76f67212 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:05.206775 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 172/346] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:01:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:01:05.207996 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625c47b6-541c-452f-942c-c68cae74789f None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:05.208414 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625c47b6-541c-452f-942c-c68cae74789f None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:05.208821 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625c47b6-541c-452f-942c-c68cae74789f None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:05.213947 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 173/347] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:01:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:01:05.214864 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625c47b6-541c-452f-942c-c68cae74789f None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:05.245842 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0f05f8-f8d5-4101-97dd-40b131f4e56e None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:05.281440 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:05.286981 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e48a234-4f67-4be7-8b1b-604942fc6212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:05.289206 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e48a234-4f67-4be7-8b1b-604942fc6212 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:05.289824 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e48a234-4f67-4be7-8b1b-604942fc6212 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:05.290241 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e48a234-4f67-4be7-8b1b-604942fc6212 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:05.318250 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:05.346055 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625c47b6-541c-452f-942c-c68cae74789f None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:05.378139 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e48a234-4f67-4be7-8b1b-604942fc6212 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:05.378139 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e48a234-4f67-4be7-8b1b-604942fc6212 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:05.394291 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e014bec5-10b8-4b8d-bffd-8cc3b6fd6c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:05.394988 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014bec5-10b8-4b8d-bffd-8cc3b6fd6c7a None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:05.395298 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014bec5-10b8-4b8d-bffd-8cc3b6fd6c7a None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:05.395812 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014bec5-10b8-4b8d-bffd-8cc3b6fd6c7a None admin] PATH_INFO: `/v3/projects/f66cc7270ac04520ad4a66086d154421` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:05.396965 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e014bec5-10b8-4b8d-bffd-8cc3b6fd6c7a None admin] RBAC: Authorizing `identity:get_project(project_id=f66cc7270ac04520ad4a66086d154421)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:05.398380 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e014bec5-10b8-4b8d-bffd-8cc3b6fd6c7a None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:05.400531 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 174/348] 10.222.0.61 () {56 vars in 1186 bytes} [Fri Aug 26 12:01:05 2022] GET /identity/v3/projects/f66cc7270ac04520ad4a66086d154421 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:05.401827 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265dd974-8001-4979-a03a-f67a58b508dc None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:05.402290 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265dd974-8001-4979-a03a-f67a58b508dc None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:05.402742 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265dd974-8001-4979-a03a-f67a58b508dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:05.424778 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-265dd974-8001-4979-a03a-f67a58b508dc None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:01:05.425970 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-265dd974-8001-4979-a03a-f67a58b508dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:01:05.505038 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 175/349] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:01:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 12:01:05.506551 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8078e576-223d-4209-bb3e-51a0537c8341 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:05.506979 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8078e576-223d-4209-bb3e-51a0537c8341 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:05.507440 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8078e576-223d-4209-bb3e-51a0537c8341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:05.513068 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 175/350] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:01:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:01:05.515041 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265dd974-8001-4979-a03a-f67a58b508dc None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:05.532415 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-8078e576-223d-4209-bb3e-51a0537c8341 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:01:05.534098 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-8078e576-223d-4209-bb3e-51a0537c8341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:01:05.561310 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:05.608294 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:05.612216 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 176/351] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:01:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:01:05.613262 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8078e576-223d-4209-bb3e-51a0537c8341 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:05.640400 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265dd974-8001-4979-a03a-f67a58b508dc None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:05.699924 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60f1284-8f3a-4838-b7ed-5fcff2ced008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:05.700579 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60f1284-8f3a-4838-b7ed-5fcff2ced008 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:05.701398 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60f1284-8f3a-4838-b7ed-5fcff2ced008 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:05.701865 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60f1284-8f3a-4838-b7ed-5fcff2ced008 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:05.716351 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60f1284-8f3a-4838-b7ed-5fcff2ced008 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:01:05.721773 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-a60f1284-8f3a-4838-b7ed-5fcff2ced008 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 12:01:05.721773 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:05.721773 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:05.721773 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:05.721773 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:05.721773 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:05.721773 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:05.721773 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:05.721773 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:05.721773 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:05.721773 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:05.721773 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:05.721773 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:01:05.721773 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:01:05.721773 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:01:05.721773 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:01:05.722689 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:01:05.722689 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:01:05.722689 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:01:05.722689 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:01:05.722689 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:05.722689 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:05.722689 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:05.722689 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:05.722689 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:05.722689 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:05.722689 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:05.722689 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:05.722689 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:05.722689 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:05.722689 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:05.722689 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:05.723557 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:05.723557 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:05.723557 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:05.723557 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:05.723557 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:01:05.723557 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:01:05.723557 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:01:05.723557 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:01:05.723557 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:01:05.723557 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:01:05.723557 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 12:01:05.723557 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:05.727675 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 176/352] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:01:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:05.728525 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60f1284-8f3a-4838-b7ed-5fcff2ced008 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:05.730218 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8078e576-223d-4209-bb3e-51a0537c8341 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:05.769537 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d30e08-2880-4790-8d49-0567d1febd5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:05.770447 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d30e08-2880-4790-8d49-0567d1febd5c None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:05.770929 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d30e08-2880-4790-8d49-0567d1febd5c None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:05.771402 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d30e08-2880-4790-8d49-0567d1febd5c None admin] PATH_INFO: `/v3/groups/8087f2b0d4ea473ab25610df0477f6ee` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:05.778231 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d30e08-2880-4790-8d49-0567d1febd5c None admin] RBAC: Authorizing `identity:get_group(group_id=8087f2b0d4ea473ab25610df0477f6ee)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:05.780662 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d30e08-2880-4790-8d49-0567d1febd5c None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:05.782500 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 177/353] 10.222.0.61 () {56 vars in 1180 bytes} [Fri Aug 26 12:01:05 2022] GET /identity/v3/groups/8087f2b0d4ea473ab25610df0477f6ee => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:05.783491 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d30e08-2880-4790-8d49-0567d1febd5c None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:05.819645 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:05.898082 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d30e08-2880-4790-8d49-0567d1febd5c None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:05.923230 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60f1284-8f3a-4838-b7ed-5fcff2ced008 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:05.939674 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7540856-c3cd-4538-8da3-8727cbba29bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:05.940583 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7540856-c3cd-4538-8da3-8727cbba29bf None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:05.941111 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7540856-c3cd-4538-8da3-8727cbba29bf None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:05.941554 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7540856-c3cd-4538-8da3-8727cbba29bf None admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:05.942397 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7540856-c3cd-4538-8da3-8727cbba29bf None admin] RBAC: Adding query filter params (name=member) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:05.942873 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7540856-c3cd-4538-8da3-8727cbba29bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:05.944463 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7540856-c3cd-4538-8da3-8727cbba29bf None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:05.953521 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 178/354] 10.222.0.61 () {56 vars in 1101 bytes} [Fri Aug 26 12:01:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:05.954600 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7540856-c3cd-4538-8da3-8727cbba29bf None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:05.980554 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd24b31-815b-4b01-b74e-97d8b500e764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:05.981333 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd24b31-815b-4b01-b74e-97d8b500e764 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:05.981679 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd24b31-815b-4b01-b74e-97d8b500e764 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:05.982473 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd24b31-815b-4b01-b74e-97d8b500e764 None admin] PATH_INFO: `/v3/projects/f66cc7270ac04520ad4a66086d154421` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:05.983691 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd24b31-815b-4b01-b74e-97d8b500e764 None admin] RBAC: Authorizing `identity:get_project(project_id=f66cc7270ac04520ad4a66086d154421)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:05.985485 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd24b31-815b-4b01-b74e-97d8b500e764 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:05.987122 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 177/355] 10.222.0.61 () {56 vars in 1186 bytes} [Fri Aug 26 12:01:05 2022] GET /identity/v3/projects/f66cc7270ac04520ad4a66086d154421 => generated 298 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:05.988111 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd24b31-815b-4b01-b74e-97d8b500e764 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:06.002770 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:06.062639 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:06.081535 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7540856-c3cd-4538-8da3-8727cbba29bf None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:06.116322 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd24b31-815b-4b01-b74e-97d8b500e764 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:06.124117 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed206a8e-3043-429e-b77c-a255b562dd2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:06.124732 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed206a8e-3043-429e-b77c-a255b562dd2f None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:06.125009 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed206a8e-3043-429e-b77c-a255b562dd2f None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:06.125281 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed206a8e-3043-429e-b77c-a255b562dd2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:06.126006 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed206a8e-3043-429e-b77c-a255b562dd2f None admin] RBAC: Adding query filter params (group.id=8087f2b0d4ea473ab25610df0477f6ee, role.id=b2403690f5ad443aad059f5ee024f61c, scope.project.id=f66cc7270ac04520ad4a66086d154421) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:06.126612 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed206a8e-3043-429e-b77c-a255b562dd2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:06.128124 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed206a8e-3043-429e-b77c-a255b562dd2f None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:06.135528 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 179/356] 10.222.0.61 () {56 vars in 1376 bytes} [Fri Aug 26 12:01:05 2022] GET /identity/v3/role_assignments?group.id=8087f2b0d4ea473ab25610df0477f6ee&scope.project.id=f66cc7270ac04520ad4a66086d154421&role.id=b2403690f5ad443aad059f5ee024f61c => generated 260 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:06.137015 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1826dd35-2a40-4a42-bdf7-59f5a4969569 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:06.137487 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1826dd35-2a40-4a42-bdf7-59f5a4969569 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:06.137765 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1826dd35-2a40-4a42-bdf7-59f5a4969569 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:06.139218 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 180/357] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:01:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:01:06.139807 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1826dd35-2a40-4a42-bdf7-59f5a4969569 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:06.164312 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fe056f-6fa6-4ebc-98d3-e8dd48850f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:06.164874 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe056f-6fa6-4ebc-98d3-e8dd48850f68 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:06.165157 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe056f-6fa6-4ebc-98d3-e8dd48850f68 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:06.165436 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe056f-6fa6-4ebc-98d3-e8dd48850f68 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:06.184227 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fe056f-6fa6-4ebc-98d3-e8dd48850f68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:01:06.189490 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-94fe056f-6fa6-4ebc-98d3-e8dd48850f68 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 12:01:06.189490 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:06.189490 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:06.189490 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:06.189490 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:06.189490 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:06.189490 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:06.189490 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:06.189490 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:06.189490 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:06.189490 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:06.189490 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:06.189490 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:01:06.189490 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:01:06.189490 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:01:06.189490 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:01:06.190443 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:01:06.190443 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:01:06.190443 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:01:06.190443 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:01:06.190443 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:06.190443 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:06.190443 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:06.190443 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:06.190443 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:06.190443 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:06.190443 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:06.190443 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:06.190443 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:06.190443 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:06.190443 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:06.190443 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:06.191291 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:06.191291 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:06.191291 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:06.191291 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:06.191291 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:01:06.191291 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:01:06.191291 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:01:06.191291 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:01:06.191291 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:01:06.191291 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:01:06.191291 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 12:01:06.191291 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:06.191291 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 178/358] 10.222.0.61 () {56 vars in 1114 bytes} [Fri Aug 26 12:01:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:06.192117 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fe056f-6fa6-4ebc-98d3-e8dd48850f68 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:06.297975 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1826dd35-2a40-4a42-bdf7-59f5a4969569 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:06.298842 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:06.330910 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fe056f-6fa6-4ebc-98d3-e8dd48850f68 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:06.340479 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768b11db-0280-4398-9b16-50e057274565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:06.341104 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768b11db-0280-4398-9b16-50e057274565 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:06.341906 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768b11db-0280-4398-9b16-50e057274565 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:06.342188 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768b11db-0280-4398-9b16-50e057274565 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:06.368687 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:06.374520 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33392c0-5312-47cb-ac8a-1bec51b146ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:06.375554 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33392c0-5312-47cb-ac8a-1bec51b146ef None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:06.375997 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33392c0-5312-47cb-ac8a-1bec51b146ef None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:06.376698 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33392c0-5312-47cb-ac8a-1bec51b146ef None admin] PATH_INFO: `/v3/groups/8087f2b0d4ea473ab25610df0477f6ee` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:06.378235 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33392c0-5312-47cb-ac8a-1bec51b146ef None admin] RBAC: Authorizing `identity:get_group(group_id=8087f2b0d4ea473ab25610df0477f6ee)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:06.380315 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33392c0-5312-47cb-ac8a-1bec51b146ef None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:06.382329 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 179/359] 10.222.0.61 () {56 vars in 1180 bytes} [Fri Aug 26 12:01:06 2022] GET /identity/v3/groups/8087f2b0d4ea473ab25610df0477f6ee => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:06.383549 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33392c0-5312-47cb-ac8a-1bec51b146ef None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:06.432863 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768b11db-0280-4398-9b16-50e057274565 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:06.434346 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768b11db-0280-4398-9b16-50e057274565 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:06.507859 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33392c0-5312-47cb-ac8a-1bec51b146ef None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:06.552369 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403c66ed-5170-44b1-88b6-9db58fb58424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:06.553005 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c66ed-5170-44b1-88b6-9db58fb58424 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:06.553284 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c66ed-5170-44b1-88b6-9db58fb58424 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:06.553552 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c66ed-5170-44b1-88b6-9db58fb58424 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:06.554659 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 181/360] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:01:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 12:01:06.559904 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a13799-e19d-4352-bd85-e75c96626028 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:06.560305 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a13799-e19d-4352-bd85-e75c96626028 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:06.560764 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a13799-e19d-4352-bd85-e75c96626028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:06.562977 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:06.570186 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403c66ed-5170-44b1-88b6-9db58fb58424 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:01:06.576834 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-403c66ed-5170-44b1-88b6-9db58fb58424 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 12:01:06.576834 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:06.576834 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:06.576834 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:06.576834 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:06.576834 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:06.576834 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:06.576834 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:06.576834 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:06.576834 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:06.576834 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:06.576834 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:06.576834 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:01:06.576834 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:01:06.576834 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:01:06.576834 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:01:06.577771 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:01:06.577771 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:01:06.577771 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:01:06.577771 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:01:06.577771 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:06.577771 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:06.577771 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:06.577771 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:06.577771 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:06.577771 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:06.577771 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:06.577771 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:06.577771 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:06.577771 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:06.577771 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:06.577771 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:06.578773 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:06.578773 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:06.578773 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:06.578773 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:06.578773 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:01:06.578773 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:01:06.578773 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:01:06.578773 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:01:06.578773 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:01:06.578773 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:01:06.578773 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 26 12:01:06.578773 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:06.581075 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 180/361] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:01:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:06.581998 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403c66ed-5170-44b1-88b6-9db58fb58424 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:06.587687 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-13a13799-e19d-4352-bd85-e75c96626028 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:01:06.589242 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-13a13799-e19d-4352-bd85-e75c96626028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:01:06.651453 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:06.681777 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 182/362] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:01:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:01:06.682780 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a13799-e19d-4352-bd85-e75c96626028 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:06.728172 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403c66ed-5170-44b1-88b6-9db58fb58424 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:06.771864 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3a9fcb-a435-4a96-8253-1bfbe23ca740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:06.773701 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3a9fcb-a435-4a96-8253-1bfbe23ca740 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:06.774149 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3a9fcb-a435-4a96-8253-1bfbe23ca740 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:06.774568 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3a9fcb-a435-4a96-8253-1bfbe23ca740 None admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:06.775664 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3a9fcb-a435-4a96-8253-1bfbe23ca740 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:06.776229 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3a9fcb-a435-4a96-8253-1bfbe23ca740 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:06.778424 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3a9fcb-a435-4a96-8253-1bfbe23ca740 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:06.790905 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 181/363] 10.222.0.61 () {56 vars in 1111 bytes} [Fri Aug 26 12:01:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:06.796350 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3a9fcb-a435-4a96-8253-1bfbe23ca740 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:06.821355 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a13799-e19d-4352-bd85-e75c96626028 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:06.863929 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94137158-6b18-46b5-beb7-43f1c05c944a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:06.864827 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94137158-6b18-46b5-beb7-43f1c05c944a None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:06.865282 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94137158-6b18-46b5-beb7-43f1c05c944a None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:06.865730 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94137158-6b18-46b5-beb7-43f1c05c944a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:06.866915 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94137158-6b18-46b5-beb7-43f1c05c944a None admin] RBAC: Adding query filter params (group.id=8087f2b0d4ea473ab25610df0477f6ee, role.id=a60f381814fe4c1da324d13c8e7d5d20, scope.project.id=f66cc7270ac04520ad4a66086d154421) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:06.867667 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94137158-6b18-46b5-beb7-43f1c05c944a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:06.869325 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94137158-6b18-46b5-beb7-43f1c05c944a None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:06.874751 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:06.875821 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 183/364] 10.222.0.61 () {56 vars in 1376 bytes} [Fri Aug 26 12:01:06 2022] GET /identity/v3/role_assignments?group.id=8087f2b0d4ea473ab25610df0477f6ee&scope.project.id=f66cc7270ac04520ad4a66086d154421&role.id=a60f381814fe4c1da324d13c8e7d5d20 => generated 260 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:06.876921 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94137158-6b18-46b5-beb7-43f1c05c944a None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:06.966176 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3a9fcb-a435-4a96-8253-1bfbe23ca740 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:06.968040 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:07.035917 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cde1533-38a0-47ca-b4c2-8ff6213a9e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:07.036792 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cde1533-38a0-47ca-b4c2-8ff6213a9e85 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:07.037276 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cde1533-38a0-47ca-b4c2-8ff6213a9e85 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:07.037689 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cde1533-38a0-47ca-b4c2-8ff6213a9e85 None admin] PATH_INFO: `/v3/projects/16904d4c978c4c12abd64d625faa3eda` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:07.039018 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cde1533-38a0-47ca-b4c2-8ff6213a9e85 None admin] RBAC: Authorizing `identity:get_project(project_id=16904d4c978c4c12abd64d625faa3eda)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:07.040737 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cde1533-38a0-47ca-b4c2-8ff6213a9e85 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:07.042738 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 182/365] 10.222.0.61 () {56 vars in 1186 bytes} [Fri Aug 26 12:01:06 2022] GET /identity/v3/projects/16904d4c978c4c12abd64d625faa3eda => generated 302 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:07.043902 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cde1533-38a0-47ca-b4c2-8ff6213a9e85 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:07.044353 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94137158-6b18-46b5-beb7-43f1c05c944a None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:07.089366 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605b9c18-faa4-45dc-9479-7e1c529f202d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:07.089976 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b9c18-faa4-45dc-9479-7e1c529f202d None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:07.090254 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b9c18-faa4-45dc-9479-7e1c529f202d None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:07.090524 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b9c18-faa4-45dc-9479-7e1c529f202d None admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:07.091668 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605b9c18-faa4-45dc-9479-7e1c529f202d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:07.091892 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605b9c18-faa4-45dc-9479-7e1c529f202d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:07.093295 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605b9c18-faa4-45dc-9479-7e1c529f202d None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:07.108037 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 184/366] 10.222.0.61 () {56 vars in 1101 bytes} [Fri Aug 26 12:01:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:07.109042 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605b9c18-faa4-45dc-9479-7e1c529f202d None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:07.158772 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cde1533-38a0-47ca-b4c2-8ff6213a9e85 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:07.187172 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:07.197038 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11870ebe-80d7-4fac-a8f0-0f2226f5880b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:07.198049 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11870ebe-80d7-4fac-a8f0-0f2226f5880b None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:07.198478 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11870ebe-80d7-4fac-a8f0-0f2226f5880b None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:07.198991 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11870ebe-80d7-4fac-a8f0-0f2226f5880b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:07.224513 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605b9c18-faa4-45dc-9479-7e1c529f202d None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:07.233015 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:07.264925 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b35445-4780-4c08-a897-7be2c0a71f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:07.265665 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b35445-4780-4c08-a897-7be2c0a71f70 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:07.266016 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b35445-4780-4c08-a897-7be2c0a71f70 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:07.266351 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b35445-4780-4c08-a897-7be2c0a71f70 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:07.278124 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b35445-4780-4c08-a897-7be2c0a71f70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:01:07.278540 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11870ebe-80d7-4fac-a8f0-0f2226f5880b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:07.281046 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11870ebe-80d7-4fac-a8f0-0f2226f5880b None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:07.285622 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-37b35445-4780-4c08-a897-7be2c0a71f70 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 12:01:07.285622 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:07.285622 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:07.285622 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:07.285622 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:07.285622 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:07.285622 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:07.285622 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:07.285622 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:07.285622 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:07.285622 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:07.285622 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:07.285622 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:01:07.285622 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:01:07.285622 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:01:07.285622 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:01:07.286757 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:01:07.286757 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:01:07.286757 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:01:07.286757 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:01:07.286757 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:07.286757 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:07.286757 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:07.286757 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:07.286757 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:07.286757 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:07.286757 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:07.286757 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:07.286757 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:07.286757 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:07.286757 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:07.286757 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:07.287770 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:07.287770 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:07.287770 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:07.287770 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:07.287770 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:01:07.287770 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:01:07.287770 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:01:07.287770 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:01:07.287770 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:01:07.287770 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:01:07.287770 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 12:01:07.287770 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:07.289337 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 185/367] 10.222.0.61 () {56 vars in 1096 bytes} [Fri Aug 26 12:01:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:07.290212 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b35445-4780-4c08-a897-7be2c0a71f70 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:07.419745 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b35445-4780-4c08-a897-7be2c0a71f70 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:07.428165 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 183/368] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:01:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 12:01:07.429585 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11870ebe-80d7-4fac-a8f0-0f2226f5880b None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:07.461873 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0753e55b-b095-4dac-a2ce-770825382c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:07.462902 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0753e55b-b095-4dac-a2ce-770825382c3c None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:07.463403 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0753e55b-b095-4dac-a2ce-770825382c3c None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:07.463895 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0753e55b-b095-4dac-a2ce-770825382c3c None admin] PATH_INFO: `/v3/groups/8087f2b0d4ea473ab25610df0477f6ee` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:07.464558 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:07.467337 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0753e55b-b095-4dac-a2ce-770825382c3c None admin] RBAC: Authorizing `identity:get_group(group_id=8087f2b0d4ea473ab25610df0477f6ee)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:07.469492 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0753e55b-b095-4dac-a2ce-770825382c3c None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:07.471767 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 186/369] 10.222.0.61 () {56 vars in 1180 bytes} [Fri Aug 26 12:01:07 2022] GET /identity/v3/groups/8087f2b0d4ea473ab25610df0477f6ee => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:07.472759 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0753e55b-b095-4dac-a2ce-770825382c3c None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:07.512341 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:07.550695 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11870ebe-80d7-4fac-a8f0-0f2226f5880b None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:07.588034 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0753e55b-b095-4dac-a2ce-770825382c3c None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:07.590461 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18c1858-bfcf-44b2-9b20-cb7e81b73d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:07.592122 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18c1858-bfcf-44b2-9b20-cb7e81b73d61 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:07.592122 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18c1858-bfcf-44b2-9b20-cb7e81b73d61 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:07.592628 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18c1858-bfcf-44b2-9b20-cb7e81b73d61 None admin] PATH_INFO: `/v3/users/f73889de61064802be83d1334ac128bc` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:07.593852 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18c1858-bfcf-44b2-9b20-cb7e81b73d61 None admin] RBAC: Authorizing `identity:get_user(user_id=f73889de61064802be83d1334ac128bc)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:07.595399 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18c1858-bfcf-44b2-9b20-cb7e81b73d61 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:07.597061 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 184/370] 10.222.0.61 () {56 vars in 1177 bytes} [Fri Aug 26 12:01:07 2022] GET /identity/v3/users/f73889de61064802be83d1334ac128bc => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:07.597882 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18c1858-bfcf-44b2-9b20-cb7e81b73d61 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:07.627382 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4f34d4-3418-46ff-b0dd-7aa7faaeb76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:07.628361 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f34d4-3418-46ff-b0dd-7aa7faaeb76d None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:07.628834 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f34d4-3418-46ff-b0dd-7aa7faaeb76d None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:07.629332 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f34d4-3418-46ff-b0dd-7aa7faaeb76d None admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:07.630280 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4f34d4-3418-46ff-b0dd-7aa7faaeb76d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:07.630899 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4f34d4-3418-46ff-b0dd-7aa7faaeb76d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:07.632701 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4f34d4-3418-46ff-b0dd-7aa7faaeb76d None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:07.642834 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 187/371] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:01:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:07.644054 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4f34d4-3418-46ff-b0dd-7aa7faaeb76d None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:07.712378 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:07.719194 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18c1858-bfcf-44b2-9b20-cb7e81b73d61 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:07.751963 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:07.758566 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4f34d4-3418-46ff-b0dd-7aa7faaeb76d None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:07.768275 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7958dfe9-c5cb-42ca-b565-324cfb0886c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:07.769633 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958dfe9-c5cb-42ca-b565-324cfb0886c8 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:07.770300 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958dfe9-c5cb-42ca-b565-324cfb0886c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:07.770778 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958dfe9-c5cb-42ca-b565-324cfb0886c8 None admin] PATH_INFO: `/v3/projects/16904d4c978c4c12abd64d625faa3eda` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:07.772126 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7958dfe9-c5cb-42ca-b565-324cfb0886c8 None admin] RBAC: Authorizing `identity:get_project(project_id=16904d4c978c4c12abd64d625faa3eda)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:07.773973 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7958dfe9-c5cb-42ca-b565-324cfb0886c8 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:07.776224 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 185/372] 10.222.0.61 () {56 vars in 1186 bytes} [Fri Aug 26 12:01:07 2022] GET /identity/v3/projects/16904d4c978c4c12abd64d625faa3eda => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:07.777366 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7958dfe9-c5cb-42ca-b565-324cfb0886c8 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:07.801234 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1303ca-1a04-4c3e-a762-070a48b0b730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:07.802110 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1303ca-1a04-4c3e-a762-070a48b0b730 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:07.802530 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1303ca-1a04-4c3e-a762-070a48b0b730 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:07.802939 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1303ca-1a04-4c3e-a762-070a48b0b730 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:07.803989 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1303ca-1a04-4c3e-a762-070a48b0b730 None admin] RBAC: Adding query filter params (group.id=8087f2b0d4ea473ab25610df0477f6ee, role.id=b2403690f5ad443aad059f5ee024f61c, scope.project.id=16904d4c978c4c12abd64d625faa3eda) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:07.804468 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1303ca-1a04-4c3e-a762-070a48b0b730 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:07.806096 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1303ca-1a04-4c3e-a762-070a48b0b730 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:07.814093 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 188/373] 10.222.0.61 () {56 vars in 1376 bytes} [Fri Aug 26 12:01:07 2022] GET /identity/v3/role_assignments?group.id=8087f2b0d4ea473ab25610df0477f6ee&scope.project.id=16904d4c978c4c12abd64d625faa3eda&role.id=b2403690f5ad443aad059f5ee024f61c => generated 260 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:07.815068 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1303ca-1a04-4c3e-a762-070a48b0b730 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:07.922733 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7958dfe9-c5cb-42ca-b565-324cfb0886c8 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:07.940547 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1303ca-1a04-4c3e-a762-070a48b0b730 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:07.983225 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac335455-40c1-4b07-853c-f3f5304bca92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:07.984438 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac335455-40c1-4b07-853c-f3f5304bca92 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:07.985361 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac335455-40c1-4b07-853c-f3f5304bca92 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:07.986136 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfad52f-b96e-410f-aa8e-ae70415f6407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:07.986897 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfad52f-b96e-410f-aa8e-ae70415f6407 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:07.987334 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac335455-40c1-4b07-853c-f3f5304bca92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:07.987837 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfad52f-b96e-410f-aa8e-ae70415f6407 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:07.988122 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfad52f-b96e-410f-aa8e-ae70415f6407 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:07.988517 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac335455-40c1-4b07-853c-f3f5304bca92 None admin] RBAC: Adding query filter params (role.id=12a2a8379bd241a098646f1c8d881614, user.id=f73889de61064802be83d1334ac128bc, scope.system=all) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:07.988958 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac335455-40c1-4b07-853c-f3f5304bca92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:07.990663 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac335455-40c1-4b07-853c-f3f5304bca92 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:08.009486 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 186/374] 10.222.0.61 () {56 vars in 1308 bytes} [Fri Aug 26 12:01:07 2022] GET /identity/v3/role_assignments?user.id=f73889de61064802be83d1334ac128bc&scope.system=all&role.id=12a2a8379bd241a098646f1c8d881614 => generated 510 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:08.010340 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac335455-40c1-4b07-853c-f3f5304bca92 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:08.056172 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:08.076464 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:08.130513 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfad52f-b96e-410f-aa8e-ae70415f6407 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:08.132021 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfad52f-b96e-410f-aa8e-ae70415f6407 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:08.225517 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac335455-40c1-4b07-853c-f3f5304bca92 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:08.255117 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 189/375] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:01:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 12:01:08.264950 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfad52f-b96e-410f-aa8e-ae70415f6407 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:08.307134 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856c4723-1353-46b8-85da-f085d08d389b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:08.307727 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c4723-1353-46b8-85da-f085d08d389b None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:08.308011 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c4723-1353-46b8-85da-f085d08d389b None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:08.308287 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c4723-1353-46b8-85da-f085d08d389b None admin] PATH_INFO: `/v3/groups/8087f2b0d4ea473ab25610df0477f6ee` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:08.309413 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856c4723-1353-46b8-85da-f085d08d389b None admin] RBAC: Authorizing `identity:get_group(group_id=8087f2b0d4ea473ab25610df0477f6ee)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:08.311548 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856c4723-1353-46b8-85da-f085d08d389b None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:08.313145 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 187/376] 10.222.0.61 () {56 vars in 1180 bytes} [Fri Aug 26 12:01:08 2022] GET /identity/v3/groups/8087f2b0d4ea473ab25610df0477f6ee => generated 226 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:08.320176 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856c4723-1353-46b8-85da-f085d08d389b None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:08.342616 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:08.426817 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfad52f-b96e-410f-aa8e-ae70415f6407 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:08.475555 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc802d0-0052-41d9-85f8-ba8c0575619b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:08.476536 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc802d0-0052-41d9-85f8-ba8c0575619b None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:08.477008 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc802d0-0052-41d9-85f8-ba8c0575619b None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:08.477513 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc802d0-0052-41d9-85f8-ba8c0575619b None admin] PATH_INFO: `/v3/projects/b98963fff98a48a1b4cae11131b453da` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:08.478857 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc802d0-0052-41d9-85f8-ba8c0575619b None admin] RBAC: Authorizing `identity:get_project(project_id=b98963fff98a48a1b4cae11131b453da)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:08.480523 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc802d0-0052-41d9-85f8-ba8c0575619b None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:08.483015 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 190/377] 10.222.0.61 () {56 vars in 1186 bytes} [Fri Aug 26 12:01:08 2022] GET /identity/v3/projects/b98963fff98a48a1b4cae11131b453da => generated 344 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:08.488517 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc802d0-0052-41d9-85f8-ba8c0575619b None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:08.509197 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:08.553393 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856c4723-1353-46b8-85da-f085d08d389b None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:08.598376 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea713c1f-999c-49b3-80b9-18ba84ec9afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:08.598857 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea713c1f-999c-49b3-80b9-18ba84ec9afb None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:08.599134 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea713c1f-999c-49b3-80b9-18ba84ec9afb None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:08.599433 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea713c1f-999c-49b3-80b9-18ba84ec9afb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:08.600163 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea713c1f-999c-49b3-80b9-18ba84ec9afb None admin] RBAC: Adding query filter params (group.id=8087f2b0d4ea473ab25610df0477f6ee, role.id=a60f381814fe4c1da324d13c8e7d5d20, scope.project.id=16904d4c978c4c12abd64d625faa3eda) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:08.600469 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea713c1f-999c-49b3-80b9-18ba84ec9afb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:08.601903 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea713c1f-999c-49b3-80b9-18ba84ec9afb None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:08.608648 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 188/378] 10.222.0.61 () {56 vars in 1376 bytes} [Fri Aug 26 12:01:08 2022] GET /identity/v3/role_assignments?group.id=8087f2b0d4ea473ab25610df0477f6ee&scope.project.id=16904d4c978c4c12abd64d625faa3eda&role.id=a60f381814fe4c1da324d13c8e7d5d20 => generated 260 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:01:08.609968 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c105bdf3-04e7-40d0-b755-c736a92c140b None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:08.610258 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c105bdf3-04e7-40d0-b755-c736a92c140b None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:08.610532 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c105bdf3-04e7-40d0-b755-c736a92c140b None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:08.611827 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 189/379] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:01:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:01:08.617722 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8872b77e-0e27-4ae3-9760-30b4c128b213 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:08.618035 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8872b77e-0e27-4ae3-9760-30b4c128b213 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:08.618275 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8872b77e-0e27-4ae3-9760-30b4c128b213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:08.622314 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc802d0-0052-41d9-85f8-ba8c0575619b None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:08.638807 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-8872b77e-0e27-4ae3-9760-30b4c128b213 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:01:08.640064 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-8872b77e-0e27-4ae3-9760-30b4c128b213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:01:08.660761 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:08.672907 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4c8f18-65cc-472b-a97a-c0342f87a2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:08.674083 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c8f18-65cc-472b-a97a-c0342f87a2a9 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:08.674528 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c8f18-65cc-472b-a97a-c0342f87a2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:08.675128 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c8f18-65cc-472b-a97a-c0342f87a2a9 None admin] PATH_INFO: `/v3/groups/722732345d594c50b8da3ecf295e6072` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:08.681521 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4c8f18-65cc-472b-a97a-c0342f87a2a9 None admin] RBAC: Authorizing `identity:get_group(group_id=722732345d594c50b8da3ecf295e6072)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:08.683159 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4c8f18-65cc-472b-a97a-c0342f87a2a9 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:08.686974 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 191/380] 10.222.0.61 () {56 vars in 1180 bytes} [Fri Aug 26 12:01:08 2022] GET /identity/v3/groups/722732345d594c50b8da3ecf295e6072 => generated 229 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:08.692617 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4c8f18-65cc-472b-a97a-c0342f87a2a9 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:08.728511 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 190/381] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:01:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 12:01:08.814557 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4c8f18-65cc-472b-a97a-c0342f87a2a9 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:08.851365 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570ce689-c314-4b67-a419-0f9df36c3dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:08.852276 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570ce689-c314-4b67-a419-0f9df36c3dcd None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:08.852708 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570ce689-c314-4b67-a419-0f9df36c3dcd None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:08.853171 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570ce689-c314-4b67-a419-0f9df36c3dcd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:08.854125 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570ce689-c314-4b67-a419-0f9df36c3dcd None admin] RBAC: Adding query filter params (group.id=722732345d594c50b8da3ecf295e6072, role.id=70ec1ba484714c8ca3adb8fe5845107c, scope.project.id=b98963fff98a48a1b4cae11131b453da) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:08.854583 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570ce689-c314-4b67-a419-0f9df36c3dcd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:08.856217 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570ce689-c314-4b67-a419-0f9df36c3dcd None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:08.864179 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 192/382] 10.222.0.61 () {56 vars in 1376 bytes} [Fri Aug 26 12:01:08 2022] GET /identity/v3/role_assignments?group.id=722732345d594c50b8da3ecf295e6072&scope.project.id=b98963fff98a48a1b4cae11131b453da&role.id=70ec1ba484714c8ca3adb8fe5845107c => generated 260 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:01:09.030913 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8872b77e-0e27-4ae3-9760-30b4c128b213 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:09.121432 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:09.140623 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8872b77e-0e27-4ae3-9760-30b4c128b213 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:09.184860 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934827b9-f3ef-4bf6-b7ba-8e403ee12a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:09.185962 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934827b9-f3ef-4bf6-b7ba-8e403ee12a48 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:09.186443 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934827b9-f3ef-4bf6-b7ba-8e403ee12a48 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:09.186925 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934827b9-f3ef-4bf6-b7ba-8e403ee12a48 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:09.198799 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934827b9-f3ef-4bf6-b7ba-8e403ee12a48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:01:09.204058 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-934827b9-f3ef-4bf6-b7ba-8e403ee12a48 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 12:01:09.204058 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:09.204058 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:09.204058 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:09.204058 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:09.204058 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:09.204058 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:09.204058 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:09.204058 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:09.204058 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:09.204058 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:09.204058 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:09.204058 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:01:09.204058 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:01:09.204058 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:01:09.204058 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:01:09.204993 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:01:09.204993 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:01:09.204993 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:01:09.204993 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:01:09.204993 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:09.204993 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:09.204993 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:09.204993 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:09.204993 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:09.204993 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:09.204993 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:09.204993 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:09.204993 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:09.204993 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:09.204993 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:09.204993 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:09.205813 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:09.205813 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:09.205813 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:09.205813 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:09.205813 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:01:09.205813 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:01:09.205813 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:01:09.205813 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:01:09.205813 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:01:09.205813 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:01:09.205813 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 12:01:09.205813 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:09.208148 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 191/383] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:01:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:09.214035 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570ce689-c314-4b67-a419-0f9df36c3dcd None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:09.280541 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:09.338369 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570ce689-c314-4b67-a419-0f9df36c3dcd None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:09.380128 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba2d83c-ee7e-4932-abe9-46ae6bbf51db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:09.381088 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba2d83c-ee7e-4932-abe9-46ae6bbf51db None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:09.381582 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba2d83c-ee7e-4932-abe9-46ae6bbf51db None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:09.382076 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba2d83c-ee7e-4932-abe9-46ae6bbf51db None admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:09.382954 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba2d83c-ee7e-4932-abe9-46ae6bbf51db None admin] RBAC: Adding query filter params (name=member) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:09.383742 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba2d83c-ee7e-4932-abe9-46ae6bbf51db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:09.385356 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba2d83c-ee7e-4932-abe9-46ae6bbf51db None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:09.399313 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 193/384] 10.222.0.61 () {56 vars in 1101 bytes} [Fri Aug 26 12:01:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:09.405675 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934827b9-f3ef-4bf6-b7ba-8e403ee12a48 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:09.542750 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934827b9-f3ef-4bf6-b7ba-8e403ee12a48 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:09.589646 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfb436a-3886-4449-aa01-410e8a8317f3 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:09.590298 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfb436a-3886-4449-aa01-410e8a8317f3 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:09.590720 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283b1c90-c2ae-42e3-82ec-caa8fe4fa8a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:09.591093 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfb436a-3886-4449-aa01-410e8a8317f3 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:09.591977 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283b1c90-c2ae-42e3-82ec-caa8fe4fa8a9 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:09.592401 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283b1c90-c2ae-42e3-82ec-caa8fe4fa8a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:09.592670 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283b1c90-c2ae-42e3-82ec-caa8fe4fa8a9 None admin] PATH_INFO: `/v3/groups/8087f2b0d4ea473ab25610df0477f6ee` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:09.594188 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 194/385] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:01:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:01:09.598022 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283b1c90-c2ae-42e3-82ec-caa8fe4fa8a9 None admin] RBAC: Authorizing `identity:get_group(group_id=8087f2b0d4ea473ab25610df0477f6ee)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:09.598022 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283b1c90-c2ae-42e3-82ec-caa8fe4fa8a9 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:09.599644 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 192/386] 10.222.0.61 () {56 vars in 1180 bytes} [Fri Aug 26 12:01:09 2022] GET /identity/v3/groups/8087f2b0d4ea473ab25610df0477f6ee => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:01:09.600084 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5210b3-9cad-4587-969a-39c080c79502 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:09.600634 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5210b3-9cad-4587-969a-39c080c79502 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:09.601130 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5210b3-9cad-4587-969a-39c080c79502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:09.607494 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283b1c90-c2ae-42e3-82ec-caa8fe4fa8a9 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:09.630246 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:09.635678 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-bc5210b3-9cad-4587-969a-39c080c79502 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:01:09.639365 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-bc5210b3-9cad-4587-969a-39c080c79502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:01:09.791896 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 195/387] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:01:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 12:01:09.797497 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283b1c90-c2ae-42e3-82ec-caa8fe4fa8a9 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:09.838653 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82076b01-8f4d-4546-b93e-c574159c99ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:09.839626 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82076b01-8f4d-4546-b93e-c574159c99ce None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:09.840114 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82076b01-8f4d-4546-b93e-c574159c99ce None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:09.840581 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82076b01-8f4d-4546-b93e-c574159c99ce None admin] PATH_INFO: `/v3/projects/f66cc7270ac04520ad4a66086d154421` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:09.847890 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82076b01-8f4d-4546-b93e-c574159c99ce None admin] RBAC: Authorizing `identity:get_project(project_id=f66cc7270ac04520ad4a66086d154421)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:09.849551 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82076b01-8f4d-4546-b93e-c574159c99ce None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:09.851419 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 193/388] 10.222.0.61 () {56 vars in 1186 bytes} [Fri Aug 26 12:01:09 2022] GET /identity/v3/projects/f66cc7270ac04520ad4a66086d154421 => generated 298 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:09.857454 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5210b3-9cad-4587-969a-39c080c79502 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:09.925020 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:09.990055 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5210b3-9cad-4587-969a-39c080c79502 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:10.034720 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a792e196-3c21-4316-888a-1e5ffcd2bdd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:10.035415 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a792e196-3c21-4316-888a-1e5ffcd2bdd4 None admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:10.035702 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a792e196-3c21-4316-888a-1e5ffcd2bdd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:10.035981 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a792e196-3c21-4316-888a-1e5ffcd2bdd4 None admin] PATH_INFO: `/v3/projects/f66cc7270ac04520ad4a66086d154421/groups/8087f2b0d4ea473ab25610df0477f6ee/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:10.037935 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a792e196-3c21-4316-888a-1e5ffcd2bdd4 None admin] RBAC: Authorizing `identity:create_grant(project_id=f66cc7270ac04520ad4a66086d154421, group_id=8087f2b0d4ea473ab25610df0477f6ee, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:10.039414 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a792e196-3c21-4316-888a-1e5ffcd2bdd4 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:10.063939 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 196/389] 10.222.0.61 () {58 vars in 1442 bytes} [Fri Aug 26 12:01:09 2022] PUT /identity/v3/projects/f66cc7270ac04520ad4a66086d154421/groups/8087f2b0d4ea473ab25610df0477f6ee/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:01:10.212424 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82076b01-8f4d-4546-b93e-c574159c99ce None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:10.316757 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:10.339832 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82076b01-8f4d-4546-b93e-c574159c99ce None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:10.379655 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b117885f-6f46-4668-9ff3-4358fb4eca5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:10.380305 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117885f-6f46-4668-9ff3-4358fb4eca5b None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:10.380591 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117885f-6f46-4668-9ff3-4358fb4eca5b None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:10.380867 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117885f-6f46-4668-9ff3-4358fb4eca5b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:10.393712 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b117885f-6f46-4668-9ff3-4358fb4eca5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:01:10.399237 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-b117885f-6f46-4668-9ff3-4358fb4eca5b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 12:01:10.399237 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:10.399237 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:10.399237 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:10.399237 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:10.399237 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:10.399237 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:10.399237 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:10.399237 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:10.399237 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:10.399237 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:10.399237 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:10.399237 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:01:10.399237 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:01:10.399237 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:01:10.399237 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:01:10.400181 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:01:10.400181 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:01:10.400181 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:01:10.400181 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:01:10.400181 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:10.400181 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:10.400181 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:10.400181 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:10.400181 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:10.400181 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:10.400181 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:10.400181 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:10.400181 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:10.400181 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:10.400181 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:10.400181 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:10.401001 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:10.401001 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:10.401001 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:10.401001 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:10.401001 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:01:10.401001 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:01:10.401001 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:01:10.401001 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:01:10.401001 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:01:10.401001 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:01:10.401001 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 12:01:10.401001 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:10.401001 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 194/390] 10.222.0.61 () {56 vars in 1114 bytes} [Fri Aug 26 12:01:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:10.407596 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a792e196-3c21-4316-888a-1e5ffcd2bdd4 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:10.516566 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a792e196-3c21-4316-888a-1e5ffcd2bdd4 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:10.528298 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da7fd72-1e83-46da-8a98-5c0a0a6c224a None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:10.528587 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da7fd72-1e83-46da-8a98-5c0a0a6c224a None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:10.529382 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da7fd72-1e83-46da-8a98-5c0a0a6c224a None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:10.531120 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 195/391] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:01:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:01:10.532760 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:10.536157 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f48121-e14d-4eaf-8e16-b2459301e7f6 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:10.536446 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f48121-e14d-4eaf-8e16-b2459301e7f6 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:10.536724 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f48121-e14d-4eaf-8e16-b2459301e7f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:10.563359 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49cbece-3ca3-441e-b35f-0bda85c6f84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:10.563985 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49cbece-3ca3-441e-b35f-0bda85c6f84a None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:10.564276 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49cbece-3ca3-441e-b35f-0bda85c6f84a None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:10.564556 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49cbece-3ca3-441e-b35f-0bda85c6f84a None admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:10.565232 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49cbece-3ca3-441e-b35f-0bda85c6f84a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:10.565544 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49cbece-3ca3-441e-b35f-0bda85c6f84a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:10.567067 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49cbece-3ca3-441e-b35f-0bda85c6f84a None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:10.577649 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 197/392] 10.222.0.61 () {56 vars in 1111 bytes} [Fri Aug 26 12:01:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:10.580209 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-a8f48121-e14d-4eaf-8e16-b2459301e7f6 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:01:10.581615 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-a8f48121-e14d-4eaf-8e16-b2459301e7f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:01:10.586729 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49cbece-3ca3-441e-b35f-0bda85c6f84a None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:10.694532 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 196/393] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:01:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:01:10.731779 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49cbece-3ca3-441e-b35f-0bda85c6f84a None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:10.786082 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14c5172-9d8a-4be0-923e-88288c267832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:10.786641 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c5172-9d8a-4be0-923e-88288c267832 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:10.786921 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c5172-9d8a-4be0-923e-88288c267832 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:10.787469 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c5172-9d8a-4be0-923e-88288c267832 None admin] PATH_INFO: `/v3/groups/8087f2b0d4ea473ab25610df0477f6ee` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:10.790018 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14c5172-9d8a-4be0-923e-88288c267832 None admin] RBAC: Authorizing `identity:get_group(group_id=8087f2b0d4ea473ab25610df0477f6ee)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:10.791569 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14c5172-9d8a-4be0-923e-88288c267832 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:10.793520 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 198/394] 10.222.0.61 () {56 vars in 1180 bytes} [Fri Aug 26 12:01:10 2022] GET /identity/v3/groups/8087f2b0d4ea473ab25610df0477f6ee => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:01:10.799389 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f48121-e14d-4eaf-8e16-b2459301e7f6 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:10.882848 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:10.923753 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f48121-e14d-4eaf-8e16-b2459301e7f6 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:10.980908 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0d9c33-3915-4d25-a374-a8e5bdcd497f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:10.981778 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0d9c33-3915-4d25-a374-a8e5bdcd497f None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:10.982744 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0d9c33-3915-4d25-a374-a8e5bdcd497f None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:10.983160 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0d9c33-3915-4d25-a374-a8e5bdcd497f None admin] PATH_INFO: `/v3/projects/f66cc7270ac04520ad4a66086d154421` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:10.984475 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0d9c33-3915-4d25-a374-a8e5bdcd497f None admin] RBAC: Authorizing `identity:get_project(project_id=f66cc7270ac04520ad4a66086d154421)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:10.986054 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0d9c33-3915-4d25-a374-a8e5bdcd497f None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:10.988154 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 197/395] 10.222.0.61 () {56 vars in 1186 bytes} [Fri Aug 26 12:01:10 2022] GET /identity/v3/projects/f66cc7270ac04520ad4a66086d154421 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:10.993612 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14c5172-9d8a-4be0-923e-88288c267832 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:11.036369 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0d9c33-3915-4d25-a374-a8e5bdcd497f None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:11.091506 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:11.156932 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0d9c33-3915-4d25-a374-a8e5bdcd497f None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:11.195546 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a292577-c96e-44b3-ba7d-fec850a43b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:11.196820 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a292577-c96e-44b3-ba7d-fec850a43b17 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:11.197300 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a292577-c96e-44b3-ba7d-fec850a43b17 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:11.197783 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a292577-c96e-44b3-ba7d-fec850a43b17 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:11.209243 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a292577-c96e-44b3-ba7d-fec850a43b17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:01:11.214870 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-2a292577-c96e-44b3-ba7d-fec850a43b17 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 12:01:11.214870 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:11.214870 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:11.214870 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:11.214870 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:11.214870 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:11.214870 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:11.214870 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:11.214870 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:11.214870 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:11.214870 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:11.214870 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:11.214870 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:01:11.214870 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:01:11.214870 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:01:11.214870 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:01:11.216203 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:01:11.216203 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:01:11.216203 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:01:11.216203 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:01:11.216203 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:11.216203 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:11.216203 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:11.216203 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:11.216203 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:11.216203 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:11.216203 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:11.216203 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:11.216203 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:11.216203 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:11.216203 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:11.216203 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:11.217479 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:11.217479 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:11.217479 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:11.217479 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:11.217479 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:01:11.217479 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:01:11.217479 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:01:11.217479 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:01:11.217479 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:01:11.217479 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:01:11.217479 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 12:01:11.217479 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:11.220629 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 198/396] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:01:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:11.220809 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:11.227176 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a292577-c96e-44b3-ba7d-fec850a43b17 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:11.235826 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14c5172-9d8a-4be0-923e-88288c267832 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:11.337984 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a292577-c96e-44b3-ba7d-fec850a43b17 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:11.349718 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caed66d-13ff-42b4-97bc-081f53b6e461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:11.350451 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caed66d-13ff-42b4-97bc-081f53b6e461 None admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:11.350741 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caed66d-13ff-42b4-97bc-081f53b6e461 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:11.351018 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caed66d-13ff-42b4-97bc-081f53b6e461 None admin] PATH_INFO: `/v3/projects/f66cc7270ac04520ad4a66086d154421/groups/8087f2b0d4ea473ab25610df0477f6ee/roles/a60f381814fe4c1da324d13c8e7d5d20` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:11.359072 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caed66d-13ff-42b4-97bc-081f53b6e461 None admin] RBAC: Authorizing `identity:create_grant(project_id=f66cc7270ac04520ad4a66086d154421, group_id=8087f2b0d4ea473ab25610df0477f6ee, role_id=a60f381814fe4c1da324d13c8e7d5d20)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:11.360592 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caed66d-13ff-42b4-97bc-081f53b6e461 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:11.378005 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f93f5db-a132-413a-be25-c033a5a5534e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:11.378951 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f93f5db-a132-413a-be25-c033a5a5534e None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:11.379466 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f93f5db-a132-413a-be25-c033a5a5534e None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:11.380060 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f93f5db-a132-413a-be25-c033a5a5534e None admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:11.380951 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f93f5db-a132-413a-be25-c033a5a5534e None admin] RBAC: Adding query filter params (name=member) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:11.381414 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f93f5db-a132-413a-be25-c033a5a5534e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:11.383633 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f93f5db-a132-413a-be25-c033a5a5534e None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:11.390711 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 199/397] 10.222.0.61 () {58 vars in 1442 bytes} [Fri Aug 26 12:01:10 2022] PUT /identity/v3/projects/f66cc7270ac04520ad4a66086d154421/groups/8087f2b0d4ea473ab25610df0477f6ee/roles/a60f381814fe4c1da324d13c8e7d5d20 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:01:11.392000 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf36fa-51e6-434e-8e16-821bd613f2d6 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:11.392286 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf36fa-51e6-434e-8e16-821bd613f2d6 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:11.392568 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf36fa-51e6-434e-8e16-821bd613f2d6 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:11.398900 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 200/399] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:01:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:01:11.399251 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 199/399] 10.222.0.61 () {56 vars in 1101 bytes} [Fri Aug 26 12:01:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:01:11.399884 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf36fa-51e6-434e-8e16-821bd613f2d6 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:11.406120 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73440783-d9f3-4a87-884c-af90024f6f6a None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:11.407014 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73440783-d9f3-4a87-884c-af90024f6f6a None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:11.407579 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73440783-d9f3-4a87-884c-af90024f6f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:11.431498 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-73440783-d9f3-4a87-884c-af90024f6f6a None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:01:11.433431 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-73440783-d9f3-4a87-884c-af90024f6f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:01:11.511618 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:11.520324 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 200/400] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:01:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 12:01:11.536082 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e095c8-dec7-4ec4-bfca-76ab78ee9995 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:11.536421 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e095c8-dec7-4ec4-bfca-76ab78ee9995 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:11.536705 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e095c8-dec7-4ec4-bfca-76ab78ee9995 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:11.542542 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 201/401] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:01:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:01:11.549476 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb22527b-2572-41e3-a065-a15d0f38379a None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:11.549813 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb22527b-2572-41e3-a065-a15d0f38379a None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:11.550028 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb22527b-2572-41e3-a065-a15d0f38379a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:11.586054 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-fb22527b-2572-41e3-a065-a15d0f38379a None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:01:11.587268 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-fb22527b-2572-41e3-a065-a15d0f38379a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:01:11.634233 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:11.641269 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf36fa-51e6-434e-8e16-821bd613f2d6 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:11.663835 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 202/402] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:01:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 12:01:11.680749 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bc005f-714f-4137-bf35-57d6ac831982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:11.681704 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc005f-714f-4137-bf35-57d6ac831982 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:11.682206 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc005f-714f-4137-bf35-57d6ac831982 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:11.682687 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc005f-714f-4137-bf35-57d6ac831982 None admin] PATH_INFO: `/v3/groups/8087f2b0d4ea473ab25610df0477f6ee` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:11.684059 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bc005f-714f-4137-bf35-57d6ac831982 None admin] RBAC: Authorizing `identity:get_group(group_id=8087f2b0d4ea473ab25610df0477f6ee)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:11.685828 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bc005f-714f-4137-bf35-57d6ac831982 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:11.687961 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 201/403] 10.222.0.61 () {56 vars in 1180 bytes} [Fri Aug 26 12:01:11 2022] GET /identity/v3/groups/8087f2b0d4ea473ab25610df0477f6ee => generated 226 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:11.696147 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb22527b-2572-41e3-a065-a15d0f38379a None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:11.818452 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb22527b-2572-41e3-a065-a15d0f38379a None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:11.841069 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:11.856693 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7cce7d-4e8c-4c2a-a327-3683d45b89b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:11.857387 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7cce7d-4e8c-4c2a-a327-3683d45b89b3 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:11.857668 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7cce7d-4e8c-4c2a-a327-3683d45b89b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:11.857940 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7cce7d-4e8c-4c2a-a327-3683d45b89b3 None admin] PATH_INFO: `/v3/projects/16904d4c978c4c12abd64d625faa3eda` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:11.859116 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7cce7d-4e8c-4c2a-a327-3683d45b89b3 None admin] RBAC: Authorizing `identity:get_project(project_id=16904d4c978c4c12abd64d625faa3eda)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:11.860597 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7cce7d-4e8c-4c2a-a327-3683d45b89b3 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:11.862529 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 203/404] 10.222.0.61 () {56 vars in 1186 bytes} [Fri Aug 26 12:01:11 2022] GET /identity/v3/projects/16904d4c978c4c12abd64d625faa3eda => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:11.867854 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bc005f-714f-4137-bf35-57d6ac831982 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:11.879242 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7cce7d-4e8c-4c2a-a327-3683d45b89b3 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:12.002973 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bc005f-714f-4137-bf35-57d6ac831982 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:12.034327 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7cce7d-4e8c-4c2a-a327-3683d45b89b3 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:12.039982 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d81492e-fd5a-413f-8b9d-fbb77217ffd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:12.040925 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d81492e-fd5a-413f-8b9d-fbb77217ffd1 None admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:12.041405 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d81492e-fd5a-413f-8b9d-fbb77217ffd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:12.041988 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d81492e-fd5a-413f-8b9d-fbb77217ffd1 None admin] PATH_INFO: `/v3/projects/16904d4c978c4c12abd64d625faa3eda/groups/8087f2b0d4ea473ab25610df0477f6ee/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:12.044266 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d81492e-fd5a-413f-8b9d-fbb77217ffd1 None admin] RBAC: Authorizing `identity:create_grant(project_id=16904d4c978c4c12abd64d625faa3eda, group_id=8087f2b0d4ea473ab25610df0477f6ee, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:12.047168 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d81492e-fd5a-413f-8b9d-fbb77217ffd1 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:12.069862 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 202/405] 10.222.0.61 () {58 vars in 1442 bytes} [Fri Aug 26 12:01:11 2022] PUT /identity/v3/projects/16904d4c978c4c12abd64d625faa3eda/groups/8087f2b0d4ea473ab25610df0477f6ee/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:01:12.070512 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d81492e-fd5a-413f-8b9d-fbb77217ffd1 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:12.096939 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf8bc9b-ff1d-4eb5-9a5a-d75b6e38079d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:12.097648 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf8bc9b-ff1d-4eb5-9a5a-d75b6e38079d None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:12.097926 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf8bc9b-ff1d-4eb5-9a5a-d75b6e38079d None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:12.098195 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf8bc9b-ff1d-4eb5-9a5a-d75b6e38079d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:12.122864 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf8bc9b-ff1d-4eb5-9a5a-d75b6e38079d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:01:12.127598 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:12.131528 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-2bf8bc9b-ff1d-4eb5-9a5a-d75b6e38079d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 12:01:12.131528 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:12.131528 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:12.131528 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:12.131528 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:12.131528 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:12.131528 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:12.131528 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:12.131528 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:12.131528 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:12.131528 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:12.131528 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:12.131528 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:01:12.131528 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:01:12.131528 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:01:12.131528 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:01:12.132485 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:01:12.132485 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:01:12.132485 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:01:12.132485 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:01:12.132485 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:12.132485 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:12.132485 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:12.132485 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:12.132485 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:12.132485 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:12.132485 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:12.132485 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:12.132485 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:12.132485 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:12.132485 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:12.132485 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:12.133545 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:12.133545 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:12.133545 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:12.133545 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:12.133545 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:01:12.133545 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:01:12.133545 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:01:12.133545 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:01:12.133545 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:01:12.133545 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:01:12.133545 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 12:01:12.133545 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:12.135712 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 204/406] 10.222.0.61 () {56 vars in 1114 bytes} [Fri Aug 26 12:01:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:12.141016 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf8bc9b-ff1d-4eb5-9a5a-d75b6e38079d None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:12.239908 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:12.267178 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d81492e-fd5a-413f-8b9d-fbb77217ffd1 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:12.306154 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf8bc9b-ff1d-4eb5-9a5a-d75b6e38079d None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:12.313975 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dbd3c1-4099-4ffa-b1e3-80f9662c4cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:12.314612 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dbd3c1-4099-4ffa-b1e3-80f9662c4cce None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:12.314890 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dbd3c1-4099-4ffa-b1e3-80f9662c4cce None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:12.315165 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dbd3c1-4099-4ffa-b1e3-80f9662c4cce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:12.328978 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dbd3c1-4099-4ffa-b1e3-80f9662c4cce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:01:12.334557 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-f5dbd3c1-4099-4ffa-b1e3-80f9662c4cce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 12:01:12.334557 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:12.334557 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:12.334557 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:12.334557 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:12.334557 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:12.334557 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:12.334557 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:12.334557 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:12.334557 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:12.334557 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:12.334557 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:12.334557 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:01:12.334557 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:01:12.334557 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:01:12.334557 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:01:12.335495 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:01:12.335495 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:01:12.335495 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:01:12.335495 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:01:12.335495 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:12.335495 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:12.335495 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:12.335495 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:12.335495 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:12.335495 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:12.335495 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:12.335495 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:12.335495 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:12.335495 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:12.335495 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:12.335495 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:12.336298 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:12.336298 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:12.336298 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:12.336298 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:12.336298 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:01:12.336298 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:01:12.336298 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:01:12.336298 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:01:12.336298 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:01:12.336298 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:01:12.336298 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 12:01:12.336298 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:12.336923 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 203/407] 10.222.0.61 () {56 vars in 1096 bytes} [Fri Aug 26 12:01:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:12.342302 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dbd3c1-4099-4ffa-b1e3-80f9662c4cce None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:12.360080 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d222a5fa-f26b-4f31-83d6-c12fdc14d2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:12.360771 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d222a5fa-f26b-4f31-83d6-c12fdc14d2e8 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:12.361096 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d222a5fa-f26b-4f31-83d6-c12fdc14d2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:12.361377 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d222a5fa-f26b-4f31-83d6-c12fdc14d2e8 None admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:12.362127 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d222a5fa-f26b-4f31-83d6-c12fdc14d2e8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:12.362437 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d222a5fa-f26b-4f31-83d6-c12fdc14d2e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:12.363942 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d222a5fa-f26b-4f31-83d6-c12fdc14d2e8 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:12.380654 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 205/408] 10.222.0.61 () {56 vars in 1111 bytes} [Fri Aug 26 12:01:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:12.386280 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d222a5fa-f26b-4f31-83d6-c12fdc14d2e8 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:12.441644 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:12.460732 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dbd3c1-4099-4ffa-b1e3-80f9662c4cce None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:12.504738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13988db9-1edb-40f6-8825-733d94532617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:12.505351 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13988db9-1edb-40f6-8825-733d94532617 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:12.505629 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13988db9-1edb-40f6-8825-733d94532617 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:12.505905 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13988db9-1edb-40f6-8825-733d94532617 None admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:12.506570 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13988db9-1edb-40f6-8825-733d94532617 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:12.507323 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13988db9-1edb-40f6-8825-733d94532617 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:12.509321 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13988db9-1edb-40f6-8825-733d94532617 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:12.520361 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 204/409] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:01:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:01:12.520622 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d222a5fa-f26b-4f31-83d6-c12fdc14d2e8 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:12.528266 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13988db9-1edb-40f6-8825-733d94532617 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:12.538026 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:12.582934 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d312d3c2-a6c0-491d-a6cd-b3ad65551fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:12.584738 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312d3c2-a6c0-491d-a6cd-b3ad65551fd4 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:12.585027 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312d3c2-a6c0-491d-a6cd-b3ad65551fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:12.585303 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312d3c2-a6c0-491d-a6cd-b3ad65551fd4 None admin] PATH_INFO: `/v3/groups/8087f2b0d4ea473ab25610df0477f6ee` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:12.587183 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d312d3c2-a6c0-491d-a6cd-b3ad65551fd4 None admin] RBAC: Authorizing `identity:get_group(group_id=8087f2b0d4ea473ab25610df0477f6ee)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:12.588665 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d312d3c2-a6c0-491d-a6cd-b3ad65551fd4 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:12.591867 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 206/410] 10.222.0.61 () {56 vars in 1180 bytes} [Fri Aug 26 12:01:12 2022] GET /identity/v3/groups/8087f2b0d4ea473ab25610df0477f6ee => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:12.596121 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d312d3c2-a6c0-491d-a6cd-b3ad65551fd4 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:12.680837 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13988db9-1edb-40f6-8825-733d94532617 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:12.714755 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d312d3c2-a6c0-491d-a6cd-b3ad65551fd4 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:12.716596 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d32d76-ebfb-424f-9c2d-0c3b09c71fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:12.717492 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d32d76-ebfb-424f-9c2d-0c3b09c71fcd None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:12.717987 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d32d76-ebfb-424f-9c2d-0c3b09c71fcd None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:12.718453 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d32d76-ebfb-424f-9c2d-0c3b09c71fcd None admin] PATH_INFO: `/v3/groups/722732345d594c50b8da3ecf295e6072` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:12.719818 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d32d76-ebfb-424f-9c2d-0c3b09c71fcd None admin] RBAC: Authorizing `identity:get_group(group_id=722732345d594c50b8da3ecf295e6072)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:12.721492 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d32d76-ebfb-424f-9c2d-0c3b09c71fcd None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:12.723600 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 205/411] 10.222.0.61 () {56 vars in 1180 bytes} [Fri Aug 26 12:01:12 2022] GET /identity/v3/groups/722732345d594c50b8da3ecf295e6072 => generated 229 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:12.724958 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac3f9b9-b40f-4470-abaf-0a28b45664a5 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:12.725448 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac3f9b9-b40f-4470-abaf-0a28b45664a5 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:12.725924 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac3f9b9-b40f-4470-abaf-0a28b45664a5 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:12.727683 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 206/412] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:01:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:01:12.729468 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac3f9b9-b40f-4470-abaf-0a28b45664a5 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:12.753456 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:12.768432 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05d01cf-a11f-4a63-a964-5609e1bc824e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:12.769079 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d01cf-a11f-4a63-a964-5609e1bc824e None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:12.769417 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d01cf-a11f-4a63-a964-5609e1bc824e None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:12.769633 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d01cf-a11f-4a63-a964-5609e1bc824e None admin] PATH_INFO: `/v3/projects/16904d4c978c4c12abd64d625faa3eda` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:12.770838 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05d01cf-a11f-4a63-a964-5609e1bc824e None admin] RBAC: Authorizing `identity:get_project(project_id=16904d4c978c4c12abd64d625faa3eda)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:12.772259 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05d01cf-a11f-4a63-a964-5609e1bc824e None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:12.779811 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 207/413] 10.222.0.61 () {56 vars in 1186 bytes} [Fri Aug 26 12:01:12 2022] GET /identity/v3/projects/16904d4c978c4c12abd64d625faa3eda => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:12.779811 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4f3116-58df-4e5d-9d96-7d947150eed9 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:12.779811 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4f3116-58df-4e5d-9d96-7d947150eed9 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:12.779811 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4f3116-58df-4e5d-9d96-7d947150eed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:12.796165 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-3d4f3116-58df-4e5d-9d96-7d947150eed9 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:01:12.797472 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-3d4f3116-58df-4e5d-9d96-7d947150eed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:01:12.845279 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:12.852297 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac3f9b9-b40f-4470-abaf-0a28b45664a5 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:12.876232 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 208/414] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:01:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:01:12.877157 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4f3116-58df-4e5d-9d96-7d947150eed9 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:12.890295 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9c33cd-6023-4a97-9136-e3970d3497d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:12.891492 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9c33cd-6023-4a97-9136-e3970d3497d4 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:12.892012 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9c33cd-6023-4a97-9136-e3970d3497d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:12.892736 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9c33cd-6023-4a97-9136-e3970d3497d4 None admin] PATH_INFO: `/v3/projects/b98963fff98a48a1b4cae11131b453da` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:12.894814 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9c33cd-6023-4a97-9136-e3970d3497d4 None admin] RBAC: Authorizing `identity:get_project(project_id=b98963fff98a48a1b4cae11131b453da)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:12.897126 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9c33cd-6023-4a97-9136-e3970d3497d4 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:12.899270 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 207/415] 10.222.0.61 () {56 vars in 1186 bytes} [Fri Aug 26 12:01:12 2022] GET /identity/v3/projects/b98963fff98a48a1b4cae11131b453da => generated 344 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:12.911226 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9c33cd-6023-4a97-9136-e3970d3497d4 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:13.043646 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:13.073534 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9c33cd-6023-4a97-9136-e3970d3497d4 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:13.091616 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4f3116-58df-4e5d-9d96-7d947150eed9 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:13.135535 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09817cc6-88e6-43e2-b30a-9f1f96ccb60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:13.136196 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09817cc6-88e6-43e2-b30a-9f1f96ccb60d None admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:13.136505 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09817cc6-88e6-43e2-b30a-9f1f96ccb60d None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:13.136841 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09817cc6-88e6-43e2-b30a-9f1f96ccb60d None admin] PATH_INFO: `/v3/projects/b98963fff98a48a1b4cae11131b453da/groups/722732345d594c50b8da3ecf295e6072/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:13.140416 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09817cc6-88e6-43e2-b30a-9f1f96ccb60d None admin] RBAC: Authorizing `identity:create_grant(project_id=b98963fff98a48a1b4cae11131b453da, group_id=722732345d594c50b8da3ecf295e6072, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:13.142095 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09817cc6-88e6-43e2-b30a-9f1f96ccb60d None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:13.165189 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 208/416] 10.222.0.61 () {58 vars in 1442 bytes} [Fri Aug 26 12:01:12 2022] PUT /identity/v3/projects/b98963fff98a48a1b4cae11131b453da/groups/722732345d594c50b8da3ecf295e6072/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:01:13.165788 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910abb7d-4a81-400f-91f1-35396f57fbb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:13.166722 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910abb7d-4a81-400f-91f1-35396f57fbb0 None admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:13.167151 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910abb7d-4a81-400f-91f1-35396f57fbb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:13.167577 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910abb7d-4a81-400f-91f1-35396f57fbb0 None admin] PATH_INFO: `/v3/projects/16904d4c978c4c12abd64d625faa3eda/groups/8087f2b0d4ea473ab25610df0477f6ee/roles/a60f381814fe4c1da324d13c8e7d5d20` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:13.170558 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09817cc6-88e6-43e2-b30a-9f1f96ccb60d None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:13.172674 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910abb7d-4a81-400f-91f1-35396f57fbb0 None admin] RBAC: Authorizing `identity:create_grant(project_id=16904d4c978c4c12abd64d625faa3eda, group_id=8087f2b0d4ea473ab25610df0477f6ee, role_id=a60f381814fe4c1da324d13c8e7d5d20)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:13.175643 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910abb7d-4a81-400f-91f1-35396f57fbb0 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:13.192698 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 209/417] 10.222.0.61 () {58 vars in 1442 bytes} [Fri Aug 26 12:01:12 2022] PUT /identity/v3/projects/16904d4c978c4c12abd64d625faa3eda/groups/8087f2b0d4ea473ab25610df0477f6ee/roles/a60f381814fe4c1da324d13c8e7d5d20 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:01:13.306792 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09817cc6-88e6-43e2-b30a-9f1f96ccb60d None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:13.346480 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e76d05a-1256-4959-b664-4d167f760232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:13.347692 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e76d05a-1256-4959-b664-4d167f760232 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:13.353257 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e76d05a-1256-4959-b664-4d167f760232 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:13.354019 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e76d05a-1256-4959-b664-4d167f760232 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:13.369932 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e76d05a-1256-4959-b664-4d167f760232 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:01:13.376312 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-3e76d05a-1256-4959-b664-4d167f760232 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 12:01:13.376312 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:13.376312 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:13.376312 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:13.376312 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:13.376312 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:13.376312 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:13.376312 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:13.376312 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:13.376312 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:13.376312 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:13.376312 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:13.376312 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:01:13.376312 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:01:13.376312 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:01:13.376312 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:01:13.377744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:01:13.377744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:01:13.377744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:01:13.377744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:01:13.377744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:13.377744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:13.377744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:13.377744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:13.377744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:13.377744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:13.377744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:13.377744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:13.377744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:13.377744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:13.377744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:13.377744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:13.378930 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:13.378930 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:13.378930 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:13.378930 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:13.378930 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:01:13.378930 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:01:13.378930 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:01:13.378930 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:01:13.378930 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:01:13.378930 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:01:13.378930 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 12:01:13.378930 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:13.382565 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 209/418] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:01:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:13.388880 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910abb7d-4a81-400f-91f1-35396f57fbb0 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:13.430429 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:13.502987 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910abb7d-4a81-400f-91f1-35396f57fbb0 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:13.539809 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2312f1f7-357e-401b-bae6-d2ffdd126388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:13.540768 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2312f1f7-357e-401b-bae6-d2ffdd126388 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:13.541246 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2312f1f7-357e-401b-bae6-d2ffdd126388 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:13.541715 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2312f1f7-357e-401b-bae6-d2ffdd126388 None admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:13.542626 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2312f1f7-357e-401b-bae6-d2ffdd126388 None admin] RBAC: Adding query filter params (name=member) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:13.543149 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2312f1f7-357e-401b-bae6-d2ffdd126388 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:13.545322 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2312f1f7-357e-401b-bae6-d2ffdd126388 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:13.554568 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 210/419] 10.222.0.61 () {56 vars in 1101 bytes} [Fri Aug 26 12:01:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:01:13.559820 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e76d05a-1256-4959-b664-4d167f760232 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:13.565491 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:13.667514 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e76d05a-1256-4959-b664-4d167f760232 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:13.702759 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262e64d6-c948-4236-b646-becebe53f91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:13.704209 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e64d6-c948-4236-b646-becebe53f91c None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:13.704683 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e64d6-c948-4236-b646-becebe53f91c None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:13.705168 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e64d6-c948-4236-b646-becebe53f91c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:13.775446 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262e64d6-c948-4236-b646-becebe53f91c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:13.776909 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262e64d6-c948-4236-b646-becebe53f91c None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:13.780112 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:13.881899 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 210/420] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:01:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 12:01:13.887573 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2312f1f7-357e-401b-bae6-d2ffdd126388 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:13.907417 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4801cea-ae9a-4689-be53-8eda585944bd None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:13.907741 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4801cea-ae9a-4689-be53-8eda585944bd None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:13.907973 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4801cea-ae9a-4689-be53-8eda585944bd None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:13.909445 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 211/421] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:01:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:01:13.914483 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c60f8f2-2db1-4c54-b4ea-85a92b6b7f9e None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:13.914779 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c60f8f2-2db1-4c54-b4ea-85a92b6b7f9e None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:13.915054 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c60f8f2-2db1-4c54-b4ea-85a92b6b7f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:13.946220 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-7c60f8f2-2db1-4c54-b4ea-85a92b6b7f9e None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:01:13.947430 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-7c60f8f2-2db1-4c54-b4ea-85a92b6b7f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:01:14.020049 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 212/422] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:01:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:01:14.060050 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:14.069765 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2312f1f7-357e-401b-bae6-d2ffdd126388 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:14.115953 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad767297-2be1-422b-a990-a4c79b964ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:14.116576 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad767297-2be1-422b-a990-a4c79b964ea8 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:14.116854 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad767297-2be1-422b-a990-a4c79b964ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:14.117130 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad767297-2be1-422b-a990-a4c79b964ea8 None admin] PATH_INFO: `/v3/projects/f66cc7270ac04520ad4a66086d154421` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:14.122810 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad767297-2be1-422b-a990-a4c79b964ea8 None admin] RBAC: Authorizing `identity:get_project(project_id=f66cc7270ac04520ad4a66086d154421)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:14.124283 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad767297-2be1-422b-a990-a4c79b964ea8 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:14.126075 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 211/423] 10.222.0.61 () {56 vars in 1186 bytes} [Fri Aug 26 12:01:13 2022] GET /identity/v3/projects/f66cc7270ac04520ad4a66086d154421 => generated 298 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:14.131605 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c60f8f2-2db1-4c54-b4ea-85a92b6b7f9e None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:14.199706 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:14.246288 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c60f8f2-2db1-4c54-b4ea-85a92b6b7f9e None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:14.282108 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70770cb8-e878-46ab-becc-1ee0724927c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:14.283060 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70770cb8-e878-46ab-becc-1ee0724927c2 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:14.283540 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70770cb8-e878-46ab-becc-1ee0724927c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:14.283980 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70770cb8-e878-46ab-becc-1ee0724927c2 None admin] PATH_INFO: `/v3/groups/8087f2b0d4ea473ab25610df0477f6ee` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:14.285272 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70770cb8-e878-46ab-becc-1ee0724927c2 None admin] RBAC: Authorizing `identity:get_group(group_id=8087f2b0d4ea473ab25610df0477f6ee)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:14.286864 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70770cb8-e878-46ab-becc-1ee0724927c2 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:14.289097 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 213/424] 10.222.0.61 () {56 vars in 1180 bytes} [Fri Aug 26 12:01:14 2022] GET /identity/v3/groups/8087f2b0d4ea473ab25610df0477f6ee => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:14.299893 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad767297-2be1-422b-a990-a4c79b964ea8 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:14.366241 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70770cb8-e878-46ab-becc-1ee0724927c2 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:14.441534 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad767297-2be1-422b-a990-a4c79b964ea8 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:14.508477 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3851bfa-1b20-4411-9e26-48cd30bf576c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:14.509010 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3851bfa-1b20-4411-9e26-48cd30bf576c None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:14.509290 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3851bfa-1b20-4411-9e26-48cd30bf576c None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:14.509562 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3851bfa-1b20-4411-9e26-48cd30bf576c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:14.510362 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3851bfa-1b20-4411-9e26-48cd30bf576c None admin] RBAC: Adding query filter params (group.id=8087f2b0d4ea473ab25610df0477f6ee, role.id=b2403690f5ad443aad059f5ee024f61c, scope.project.id=f66cc7270ac04520ad4a66086d154421) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:14.510668 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3851bfa-1b20-4411-9e26-48cd30bf576c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:14.513908 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3851bfa-1b20-4411-9e26-48cd30bf576c None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:14.528092 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 212/425] 10.222.0.61 () {56 vars in 1376 bytes} [Fri Aug 26 12:01:14 2022] GET /identity/v3/role_assignments?group.id=8087f2b0d4ea473ab25610df0477f6ee&scope.project.id=f66cc7270ac04520ad4a66086d154421&role.id=b2403690f5ad443aad059f5ee024f61c => generated 611 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:14.545946 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70770cb8-e878-46ab-becc-1ee0724927c2 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:14.551730 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:14.560502 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aa47ad-75ab-4711-9fd0-a0f58ac54922 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:14.561061 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aa47ad-75ab-4711-9fd0-a0f58ac54922 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:14.561061 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aa47ad-75ab-4711-9fd0-a0f58ac54922 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:14.562746 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 213/426] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:01:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 12:01:14.568707 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54884b2a-a202-4f3c-bb65-d08e165116be None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:14.569042 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54884b2a-a202-4f3c-bb65-d08e165116be None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:14.569262 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54884b2a-a202-4f3c-bb65-d08e165116be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:14.602285 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3496117-2fb3-4013-ae84-fb2830f03f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:14.602978 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3496117-2fb3-4013-ae84-fb2830f03f9e None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:14.603291 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3496117-2fb3-4013-ae84-fb2830f03f9e None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:14.603566 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3496117-2fb3-4013-ae84-fb2830f03f9e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:14.604279 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-54884b2a-a202-4f3c-bb65-d08e165116be None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:01:14.605488 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-54884b2a-a202-4f3c-bb65-d08e165116be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:01:14.617049 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3496117-2fb3-4013-ae84-fb2830f03f9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:01:14.623049 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-c3496117-2fb3-4013-ae84-fb2830f03f9e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 12:01:14.623049 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:14.623049 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:14.623049 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:14.623049 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:14.623049 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:14.623049 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:14.623049 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:14.623049 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:14.623049 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:14.623049 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:14.623049 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:14.623049 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:01:14.623049 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:01:14.623049 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:01:14.623049 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:01:14.624065 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:01:14.624065 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:01:14.624065 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:01:14.624065 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:01:14.624065 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:14.624065 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:14.624065 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:14.624065 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:14.624065 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:14.624065 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:14.624065 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:14.624065 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:14.624065 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:14.624065 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:14.624065 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:14.624065 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:14.624894 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:14.624894 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:14.624894 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:14.624894 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:14.624894 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:01:14.624894 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:01:14.624894 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:01:14.624894 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:01:14.624894 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:01:14.624894 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:01:14.624894 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 12:01:14.624894 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:14.627933 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 214/427] 10.222.0.61 () {56 vars in 1114 bytes} [Fri Aug 26 12:01:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 12:01:14.634443 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3496117-2fb3-4013-ae84-fb2830f03f9e None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:14.682206 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:14.752042 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3496117-2fb3-4013-ae84-fb2830f03f9e None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:14.791496 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 214/428] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:01:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 12:01:14.793526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b450ab66-5637-4cc4-8aa3-5afbcfbbd78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:14.794865 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b450ab66-5637-4cc4-8aa3-5afbcfbbd78d None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:14.795426 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b450ab66-5637-4cc4-8aa3-5afbcfbbd78d None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:14.795906 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b450ab66-5637-4cc4-8aa3-5afbcfbbd78d None admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:14.796763 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b450ab66-5637-4cc4-8aa3-5afbcfbbd78d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:14.797300 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b450ab66-5637-4cc4-8aa3-5afbcfbbd78d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:14.798965 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b450ab66-5637-4cc4-8aa3-5afbcfbbd78d None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:14.808152 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 215/429] 10.222.0.61 () {56 vars in 1111 bytes} [Fri Aug 26 12:01:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:14.814119 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54884b2a-a202-4f3c-bb65-d08e165116be None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:14.923469 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54884b2a-a202-4f3c-bb65-d08e165116be None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:14.965838 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54855867-0376-40cc-9c72-2eeb08a0acd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:14.966934 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54855867-0376-40cc-9c72-2eeb08a0acd7 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:14.967421 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54855867-0376-40cc-9c72-2eeb08a0acd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:14.967903 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54855867-0376-40cc-9c72-2eeb08a0acd7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:14.998387 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:15.046476 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54855867-0376-40cc-9c72-2eeb08a0acd7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:15.048310 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54855867-0376-40cc-9c72-2eeb08a0acd7 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:15.088315 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b450ab66-5637-4cc4-8aa3-5afbcfbbd78d None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:15.126209 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:15.165947 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 215/430] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:01:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 12:01:15.173468 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54855867-0376-40cc-9c72-2eeb08a0acd7 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:15.229114 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b450ab66-5637-4cc4-8aa3-5afbcfbbd78d None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:15.245227 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:15.276575 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9c972a-5076-456d-9132-79ad0c0f96cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:15.277723 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9c972a-5076-456d-9132-79ad0c0f96cd None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:15.278590 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9c972a-5076-456d-9132-79ad0c0f96cd None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:15.279108 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9c972a-5076-456d-9132-79ad0c0f96cd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:15.293360 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9c972a-5076-456d-9132-79ad0c0f96cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:01:15.297268 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54855867-0376-40cc-9c72-2eeb08a0acd7 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:15.301796 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-0c9c972a-5076-456d-9132-79ad0c0f96cd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 26 12:01:15.301796 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:15.301796 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:15.301796 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:15.301796 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:15.301796 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:15.301796 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:15.301796 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:15.301796 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:15.301796 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:15.301796 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:15.301796 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:15.301796 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:01:15.301796 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:01:15.301796 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:01:15.301796 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:01:15.302789 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:01:15.302789 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:01:15.302789 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:01:15.302789 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:01:15.302789 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:15.302789 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:15.302789 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:15.302789 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:15.302789 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:15.302789 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:15.302789 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:15.302789 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:15.302789 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:15.302789 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:15.302789 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:15.302789 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:15.304499 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:15.304499 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:15.304499 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:15.304499 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:15.304499 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:01:15.304499 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:01:15.304499 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:01:15.304499 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:01:15.304499 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:01:15.304499 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:01:15.304499 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 26 12:01:15.304499 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:15.311420 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 216/431] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:01:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:15.314739 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9c972a-5076-456d-9132-79ad0c0f96cd None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:15.344769 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7544851c-f242-4b28-833e-b056dab7ef3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:15.345414 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7544851c-f242-4b28-833e-b056dab7ef3b None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:15.345695 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7544851c-f242-4b28-833e-b056dab7ef3b None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:15.345970 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7544851c-f242-4b28-833e-b056dab7ef3b None admin] PATH_INFO: `/v3/projects/f66cc7270ac04520ad4a66086d154421` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:15.347117 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7544851c-f242-4b28-833e-b056dab7ef3b None admin] RBAC: Authorizing `identity:get_project(project_id=f66cc7270ac04520ad4a66086d154421)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:15.348737 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7544851c-f242-4b28-833e-b056dab7ef3b None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:15.350659 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 216/432] 10.222.0.61 () {56 vars in 1186 bytes} [Fri Aug 26 12:01:15 2022] GET /identity/v3/projects/f66cc7270ac04520ad4a66086d154421 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:01:15.356576 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7544851c-f242-4b28-833e-b056dab7ef3b None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:15.450658 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:15.470201 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9c972a-5076-456d-9132-79ad0c0f96cd None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:15.482918 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7544851c-f242-4b28-833e-b056dab7ef3b None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:15.509961 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7089a0c-87c9-44f9-a00f-3081a37eb8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:15.511027 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7089a0c-87c9-44f9-a00f-3081a37eb8ff None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:15.511604 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:15.512388 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7089a0c-87c9-44f9-a00f-3081a37eb8ff None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:15.512975 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7089a0c-87c9-44f9-a00f-3081a37eb8ff None admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:15.514204 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7089a0c-87c9-44f9-a00f-3081a37eb8ff None admin] RBAC: Adding query filter params (name=member) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:15.514732 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7089a0c-87c9-44f9-a00f-3081a37eb8ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:15.516419 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7089a0c-87c9-44f9-a00f-3081a37eb8ff None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:15.522641 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa04009-a187-46c2-b69d-d19f130765d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:15.523637 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa04009-a187-46c2-b69d-d19f130765d8 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:15.523637 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa04009-a187-46c2-b69d-d19f130765d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:15.523825 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa04009-a187-46c2-b69d-d19f130765d8 None admin] PATH_INFO: `/v3/groups/8087f2b0d4ea473ab25610df0477f6ee` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:15.526164 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa04009-a187-46c2-b69d-d19f130765d8 None admin] RBAC: Authorizing `identity:get_group(group_id=8087f2b0d4ea473ab25610df0477f6ee)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:15.526591 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa04009-a187-46c2-b69d-d19f130765d8 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:15.528502 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 217/433] 10.222.0.61 () {56 vars in 1180 bytes} [Fri Aug 26 12:01:15 2022] GET /identity/v3/groups/8087f2b0d4ea473ab25610df0477f6ee => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:15.529265 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 217/434] 10.222.0.61 () {56 vars in 1101 bytes} [Fri Aug 26 12:01:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:15.533598 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d4f0f-ddaf-4f05-a7ef-58df18e71d32 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:15.533889 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d4f0f-ddaf-4f05-a7ef-58df18e71d32 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:15.534168 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d4f0f-ddaf-4f05-a7ef-58df18e71d32 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:15.535290 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7089a0c-87c9-44f9-a00f-3081a37eb8ff None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:15.535706 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 218/435] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:01:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:01:15.540886 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4db0c2c-e085-462b-ae42-033245b6959b None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:15.541167 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4db0c2c-e085-462b-ae42-033245b6959b None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:15.541437 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4db0c2c-e085-462b-ae42-033245b6959b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:15.564962 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-d4db0c2c-e085-462b-ae42-033245b6959b None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:01:15.566053 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-d4db0c2c-e085-462b-ae42-033245b6959b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:01:15.647356 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 219/436] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:01:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:01:15.647356 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4db0c2c-e085-462b-ae42-033245b6959b None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:15.665091 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7089a0c-87c9-44f9-a00f-3081a37eb8ff None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:15.702303 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aeb897-6360-4a87-a9e7-406ec9d4aa17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:15.702910 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aeb897-6360-4a87-a9e7-406ec9d4aa17 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:15.703190 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aeb897-6360-4a87-a9e7-406ec9d4aa17 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:15.703483 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aeb897-6360-4a87-a9e7-406ec9d4aa17 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:15.704206 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aeb897-6360-4a87-a9e7-406ec9d4aa17 None admin] RBAC: Adding query filter params (group.id=8087f2b0d4ea473ab25610df0477f6ee, role.id=a60f381814fe4c1da324d13c8e7d5d20, scope.project.id=f66cc7270ac04520ad4a66086d154421) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:15.704511 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aeb897-6360-4a87-a9e7-406ec9d4aa17 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:15.705960 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aeb897-6360-4a87-a9e7-406ec9d4aa17 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:15.712407 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 218/437] 10.222.0.61 () {56 vars in 1376 bytes} [Fri Aug 26 12:01:15 2022] GET /identity/v3/role_assignments?group.id=8087f2b0d4ea473ab25610df0477f6ee&scope.project.id=f66cc7270ac04520ad4a66086d154421&role.id=a60f381814fe4c1da324d13c8e7d5d20 => generated 611 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:15.756747 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c00ec-0cbb-4a23-ba06-c2968043342e None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:15.757258 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c00ec-0cbb-4a23-ba06-c2968043342e None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:15.757332 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c00ec-0cbb-4a23-ba06-c2968043342e None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:15.759106 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 219/438] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:01:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:01:15.766211 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9fc901-6cb5-41ec-99bc-ca24f3c3d129 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:15.766510 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9fc901-6cb5-41ec-99bc-ca24f3c3d129 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:15.766758 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9fc901-6cb5-41ec-99bc-ca24f3c3d129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:15.788298 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4db0c2c-e085-462b-ae42-033245b6959b None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:15.793295 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-6b9fc901-6cb5-41ec-99bc-ca24f3c3d129 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:01:15.796297 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-6b9fc901-6cb5-41ec-99bc-ca24f3c3d129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:01:15.822902 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:15.834264 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13646672-297b-4686-af71-d6b6d116b33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:15.835273 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13646672-297b-4686-af71-d6b6d116b33f None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:15.835581 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:15.835952 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13646672-297b-4686-af71-d6b6d116b33f None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:15.836439 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13646672-297b-4686-af71-d6b6d116b33f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:15.890917 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 220/439] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:01:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 12:01:15.920887 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13646672-297b-4686-af71-d6b6d116b33f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:15.922337 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13646672-297b-4686-af71-d6b6d116b33f None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:15.932744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9fc901-6cb5-41ec-99bc-ca24f3c3d129 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:16.048287 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 220/440] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:01:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 12:01:16.054058 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13646672-297b-4686-af71-d6b6d116b33f None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:16.073673 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9fc901-6cb5-41ec-99bc-ca24f3c3d129 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:16.087277 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:16.120784 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f21bde-c922-428f-99d2-756df534f416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:16.121598 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f21bde-c922-428f-99d2-756df534f416 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:16.122006 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f21bde-c922-428f-99d2-756df534f416 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:16.122382 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f21bde-c922-428f-99d2-756df534f416 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:16.134386 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:16.138238 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f21bde-c922-428f-99d2-756df534f416 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:01:16.148053 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-e7f21bde-c922-428f-99d2-756df534f416 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 12:01:16.148053 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:16.148053 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:16.148053 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:16.148053 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:16.148053 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:16.148053 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:16.148053 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:16.148053 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:16.148053 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:16.148053 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:16.148053 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:16.148053 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:01:16.148053 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:01:16.148053 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:01:16.148053 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:01:16.149006 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:01:16.149006 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:01:16.149006 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:01:16.149006 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:01:16.149006 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:16.149006 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:16.149006 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:16.149006 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:16.149006 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:16.149006 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:16.149006 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:16.149006 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:16.149006 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:16.149006 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:16.149006 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:16.149006 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:16.149813 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:16.149813 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:16.149813 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:16.149813 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:16.149813 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:01:16.149813 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:01:16.149813 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:01:16.149813 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:01:16.149813 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:01:16.149813 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:01:16.149813 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 12:01:16.149813 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:16.149813 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 221/441] 10.222.0.61 () {56 vars in 1096 bytes} [Fri Aug 26 12:01:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 12:01:16.156848 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f21bde-c922-428f-99d2-756df534f416 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:16.184187 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13646672-297b-4686-af71-d6b6d116b33f None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:16.222947 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a445ed5-f521-4dcc-8395-40e99097b9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:16.223600 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a445ed5-f521-4dcc-8395-40e99097b9fb None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:16.224235 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a445ed5-f521-4dcc-8395-40e99097b9fb None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:16.224235 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a445ed5-f521-4dcc-8395-40e99097b9fb None admin] PATH_INFO: `/v3/projects/16904d4c978c4c12abd64d625faa3eda` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:16.225243 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a445ed5-f521-4dcc-8395-40e99097b9fb None admin] RBAC: Authorizing `identity:get_project(project_id=16904d4c978c4c12abd64d625faa3eda)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:16.226683 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a445ed5-f521-4dcc-8395-40e99097b9fb None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:16.228326 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 221/442] 10.222.0.61 () {56 vars in 1186 bytes} [Fri Aug 26 12:01:16 2022] GET /identity/v3/projects/16904d4c978c4c12abd64d625faa3eda => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:01:16.232111 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a445ed5-f521-4dcc-8395-40e99097b9fb None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:16.270307 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f21bde-c922-428f-99d2-756df534f416 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:16.303224 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d834cd7-ba40-4682-93a6-58d71b49479a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:16.303837 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d834cd7-ba40-4682-93a6-58d71b49479a None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:16.304115 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d834cd7-ba40-4682-93a6-58d71b49479a None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:16.304389 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d834cd7-ba40-4682-93a6-58d71b49479a None admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:16.305051 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d834cd7-ba40-4682-93a6-58d71b49479a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:16.305350 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d834cd7-ba40-4682-93a6-58d71b49479a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:16.306734 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d834cd7-ba40-4682-93a6-58d71b49479a None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:16.314669 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 222/443] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:01:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:16.315477 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d834cd7-ba40-4682-93a6-58d71b49479a None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:16.325009 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:16.333455 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a445ed5-f521-4dcc-8395-40e99097b9fb None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:16.367046 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e852d0-bf10-494e-bf25-66ec0a358755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:16.367699 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e852d0-bf10-494e-bf25-66ec0a358755 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:16.367972 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e852d0-bf10-494e-bf25-66ec0a358755 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:16.368238 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e852d0-bf10-494e-bf25-66ec0a358755 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:16.377166 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:16.378822 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e852d0-bf10-494e-bf25-66ec0a358755 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:01:16.383997 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-21e852d0-bf10-494e-bf25-66ec0a358755 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 12:01:16.383997 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:16.383997 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:16.383997 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:16.383997 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:16.383997 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:16.383997 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:16.383997 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:16.383997 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:16.383997 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:16.383997 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:16.383997 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:16.383997 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:01:16.383997 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:01:16.383997 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:01:16.383997 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:01:16.384901 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:01:16.384901 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:01:16.384901 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:01:16.384901 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:01:16.384901 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:16.384901 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:16.384901 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:16.384901 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:16.384901 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:16.384901 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:16.384901 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:16.384901 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:16.384901 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:16.384901 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:16.384901 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:16.384901 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:16.385797 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:16.385797 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:16.385797 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:16.385797 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:16.385797 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:01:16.385797 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:01:16.385797 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:01:16.385797 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:01:16.385797 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:01:16.385797 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:01:16.385797 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 12:01:16.385797 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:16.385797 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 222/444] 10.222.0.61 () {56 vars in 1114 bytes} [Fri Aug 26 12:01:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:16.386458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e852d0-bf10-494e-bf25-66ec0a358755 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:16.418322 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d834cd7-ba40-4682-93a6-58d71b49479a None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:16.450262 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a051bc4-c5f5-416f-b5ee-3cee6a563f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:16.451417 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a051bc4-c5f5-416f-b5ee-3cee6a563f47 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:16.451697 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a051bc4-c5f5-416f-b5ee-3cee6a563f47 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:16.451964 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a051bc4-c5f5-416f-b5ee-3cee6a563f47 None admin] PATH_INFO: `/v3/groups/8087f2b0d4ea473ab25610df0477f6ee` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:16.453072 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a051bc4-c5f5-416f-b5ee-3cee6a563f47 None admin] RBAC: Authorizing `identity:get_group(group_id=8087f2b0d4ea473ab25610df0477f6ee)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:16.454512 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a051bc4-c5f5-416f-b5ee-3cee6a563f47 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:16.456085 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 223/445] 10.222.0.61 () {56 vars in 1180 bytes} [Fri Aug 26 12:01:16 2022] GET /identity/v3/groups/8087f2b0d4ea473ab25610df0477f6ee => generated 226 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:16.456836 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a051bc4-c5f5-416f-b5ee-3cee6a563f47 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:16.488937 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e852d0-bf10-494e-bf25-66ec0a358755 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:16.523341 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841e0987-e614-4365-a80b-def54a24f827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:16.523941 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841e0987-e614-4365-a80b-def54a24f827 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:16.524218 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841e0987-e614-4365-a80b-def54a24f827 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:16.524487 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841e0987-e614-4365-a80b-def54a24f827 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:16.548447 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:16.553258 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a051bc4-c5f5-416f-b5ee-3cee6a563f47 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:16.581409 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:16.587853 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220befba-6188-4eb3-bc19-c9251080bf68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:16.588440 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220befba-6188-4eb3-bc19-c9251080bf68 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:16.588716 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220befba-6188-4eb3-bc19-c9251080bf68 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:16.588984 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220befba-6188-4eb3-bc19-c9251080bf68 None admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:16.589632 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220befba-6188-4eb3-bc19-c9251080bf68 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:16.589929 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220befba-6188-4eb3-bc19-c9251080bf68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:16.591309 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220befba-6188-4eb3-bc19-c9251080bf68 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:16.593397 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841e0987-e614-4365-a80b-def54a24f827 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:16.594828 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841e0987-e614-4365-a80b-def54a24f827 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:16.599607 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 224/446] 10.222.0.61 () {56 vars in 1111 bytes} [Fri Aug 26 12:01:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:16.600385 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220befba-6188-4eb3-bc19-c9251080bf68 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:16.691479 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 223/447] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:01:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 12:01:16.692319 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841e0987-e614-4365-a80b-def54a24f827 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:16.699924 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220befba-6188-4eb3-bc19-c9251080bf68 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:16.732356 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7134f3-fd48-45dc-a818-fe3c4b7df7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:16.732979 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7134f3-fd48-45dc-a818-fe3c4b7df7d0 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:16.733253 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7134f3-fd48-45dc-a818-fe3c4b7df7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:16.733519 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7134f3-fd48-45dc-a818-fe3c4b7df7d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:16.734231 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7134f3-fd48-45dc-a818-fe3c4b7df7d0 None admin] RBAC: Adding query filter params (group.id=8087f2b0d4ea473ab25610df0477f6ee, role.id=b2403690f5ad443aad059f5ee024f61c, scope.project.id=16904d4c978c4c12abd64d625faa3eda) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:16.734535 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7134f3-fd48-45dc-a818-fe3c4b7df7d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:16.735989 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7134f3-fd48-45dc-a818-fe3c4b7df7d0 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:16.745030 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 225/448] 10.222.0.61 () {56 vars in 1376 bytes} [Fri Aug 26 12:01:16 2022] GET /identity/v3/role_assignments?group.id=8087f2b0d4ea473ab25610df0477f6ee&scope.project.id=16904d4c978c4c12abd64d625faa3eda&role.id=b2403690f5ad443aad059f5ee024f61c => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:16.745834 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7134f3-fd48-45dc-a818-fe3c4b7df7d0 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:16.748748 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:16.798798 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841e0987-e614-4365-a80b-def54a24f827 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:16.812070 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:16.836727 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7f13b6-c895-40f9-b6f0-8cbe8104ccf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:16.837260 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7f13b6-c895-40f9-b6f0-8cbe8104ccf3 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:16.837537 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7f13b6-c895-40f9-b6f0-8cbe8104ccf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:16.837806 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7f13b6-c895-40f9-b6f0-8cbe8104ccf3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:16.863600 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7134f3-fd48-45dc-a818-fe3c4b7df7d0 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:16.906082 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20f79c5-b177-4e77-8017-f0a6601d94c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:16.906684 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f79c5-b177-4e77-8017-f0a6601d94c2 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:16.906962 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f79c5-b177-4e77-8017-f0a6601d94c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:16.907253 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f79c5-b177-4e77-8017-f0a6601d94c2 None admin] PATH_INFO: `/v3/projects/b98963fff98a48a1b4cae11131b453da` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:16.908366 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20f79c5-b177-4e77-8017-f0a6601d94c2 None admin] RBAC: Authorizing `identity:get_project(project_id=b98963fff98a48a1b4cae11131b453da)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:16.909806 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20f79c5-b177-4e77-8017-f0a6601d94c2 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:16.911897 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 226/449] 10.222.0.61 () {56 vars in 1186 bytes} [Fri Aug 26 12:01:16 2022] GET /identity/v3/projects/b98963fff98a48a1b4cae11131b453da => generated 344 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:16.914561 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7f13b6-c895-40f9-b6f0-8cbe8104ccf3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:16.916052 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7f13b6-c895-40f9-b6f0-8cbe8104ccf3 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:16.916972 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20f79c5-b177-4e77-8017-f0a6601d94c2 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:16.983668 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:17.037631 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 224/450] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:01:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 12:01:17.041744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20f79c5-b177-4e77-8017-f0a6601d94c2 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:17.043540 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7f13b6-c895-40f9-b6f0-8cbe8104ccf3 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:17.072308 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:17.085667 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407769f3-4f7d-468c-bdb3-b72d6a83dee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:17.087192 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407769f3-4f7d-468c-bdb3-b72d6a83dee7 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:17.087192 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407769f3-4f7d-468c-bdb3-b72d6a83dee7 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:17.087192 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407769f3-4f7d-468c-bdb3-b72d6a83dee7 None admin] PATH_INFO: `/v3/groups/722732345d594c50b8da3ecf295e6072` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:17.088745 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407769f3-4f7d-468c-bdb3-b72d6a83dee7 None admin] RBAC: Authorizing `identity:get_group(group_id=722732345d594c50b8da3ecf295e6072)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:17.090197 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407769f3-4f7d-468c-bdb3-b72d6a83dee7 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:17.092395 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 227/451] 10.222.0.61 () {56 vars in 1180 bytes} [Fri Aug 26 12:01:16 2022] GET /identity/v3/groups/722732345d594c50b8da3ecf295e6072 => generated 229 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:17.098461 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407769f3-4f7d-468c-bdb3-b72d6a83dee7 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:17.156042 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7f13b6-c895-40f9-b6f0-8cbe8104ccf3 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:17.189864 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694cc290-3f3d-4ff5-a5cd-2fbf310fc935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:17.190488 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694cc290-3f3d-4ff5-a5cd-2fbf310fc935 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:17.190767 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694cc290-3f3d-4ff5-a5cd-2fbf310fc935 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:17.191039 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694cc290-3f3d-4ff5-a5cd-2fbf310fc935 None admin] PATH_INFO: `/v3/projects/16904d4c978c4c12abd64d625faa3eda` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:17.192176 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694cc290-3f3d-4ff5-a5cd-2fbf310fc935 None admin] RBAC: Authorizing `identity:get_project(project_id=16904d4c978c4c12abd64d625faa3eda)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:17.193615 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694cc290-3f3d-4ff5-a5cd-2fbf310fc935 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:17.195415 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 225/452] 10.222.0.61 () {56 vars in 1186 bytes} [Fri Aug 26 12:01:17 2022] GET /identity/v3/projects/16904d4c978c4c12abd64d625faa3eda => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:17.198501 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407769f3-4f7d-468c-bdb3-b72d6a83dee7 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:17.200641 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694cc290-3f3d-4ff5-a5cd-2fbf310fc935 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:17.224617 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:17.232899 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d6ce86-2099-4201-827d-cb2a6ecdb7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:17.233497 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d6ce86-2099-4201-827d-cb2a6ecdb7e2 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:17.233776 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d6ce86-2099-4201-827d-cb2a6ecdb7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:17.234048 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d6ce86-2099-4201-827d-cb2a6ecdb7e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:17.234768 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d6ce86-2099-4201-827d-cb2a6ecdb7e2 None admin] RBAC: Adding query filter params (group.id=722732345d594c50b8da3ecf295e6072, role.id=70ec1ba484714c8ca3adb8fe5845107c, scope.project.id=b98963fff98a48a1b4cae11131b453da) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:17.235074 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d6ce86-2099-4201-827d-cb2a6ecdb7e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:17.236543 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d6ce86-2099-4201-827d-cb2a6ecdb7e2 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:17.242095 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 228/453] 10.222.0.61 () {56 vars in 1376 bytes} [Fri Aug 26 12:01:17 2022] GET /identity/v3/role_assignments?group.id=722732345d594c50b8da3ecf295e6072&scope.project.id=b98963fff98a48a1b4cae11131b453da&role.id=70ec1ba484714c8ca3adb8fe5845107c => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:01:17.307470 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694cc290-3f3d-4ff5-a5cd-2fbf310fc935 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:17.344527 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3033c48a-bcc2-4d99-a0a4-46c2a6ab33e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:17.345088 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033c48a-bcc2-4d99-a0a4-46c2a6ab33e7 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:17.345364 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033c48a-bcc2-4d99-a0a4-46c2a6ab33e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:17.345635 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033c48a-bcc2-4d99-a0a4-46c2a6ab33e7 None admin] PATH_INFO: `/v3/groups/8087f2b0d4ea473ab25610df0477f6ee` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:17.346770 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3033c48a-bcc2-4d99-a0a4-46c2a6ab33e7 None admin] RBAC: Authorizing `identity:get_group(group_id=8087f2b0d4ea473ab25610df0477f6ee)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:17.348269 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3033c48a-bcc2-4d99-a0a4-46c2a6ab33e7 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:17.349893 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 226/454] 10.222.0.61 () {56 vars in 1180 bytes} [Fri Aug 26 12:01:17 2022] GET /identity/v3/groups/8087f2b0d4ea473ab25610df0477f6ee => generated 226 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:01:17.355962 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d6ce86-2099-4201-827d-cb2a6ecdb7e2 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:17.415992 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:17.460372 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d6ce86-2099-4201-827d-cb2a6ecdb7e2 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:17.493287 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207c8df7-f03b-46cf-bd32-d2266ae242a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:17.493886 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c8df7-f03b-46cf-bd32-d2266ae242a6 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:17.494166 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c8df7-f03b-46cf-bd32-d2266ae242a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:17.494439 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c8df7-f03b-46cf-bd32-d2266ae242a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:17.495157 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207c8df7-f03b-46cf-bd32-d2266ae242a6 None admin] RBAC: Adding query filter params (group.id=8087f2b0d4ea473ab25610df0477f6ee, role.id=a60f381814fe4c1da324d13c8e7d5d20, scope.project.id=16904d4c978c4c12abd64d625faa3eda) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:17.495499 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207c8df7-f03b-46cf-bd32-d2266ae242a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:17.496939 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207c8df7-f03b-46cf-bd32-d2266ae242a6 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:17.502601 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 229/455] 10.222.0.61 () {56 vars in 1376 bytes} [Fri Aug 26 12:01:17 2022] GET /identity/v3/role_assignments?group.id=8087f2b0d4ea473ab25610df0477f6ee&scope.project.id=16904d4c978c4c12abd64d625faa3eda&role.id=a60f381814fe4c1da324d13c8e7d5d20 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:20.032952 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26085dd0-c167-4274-8e9d-d93bcc8439da None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:20.033457 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26085dd0-c167-4274-8e9d-d93bcc8439da None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:20.033457 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26085dd0-c167-4274-8e9d-d93bcc8439da None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:20.034742 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 227/456] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:01:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:01:20.040576 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ad9c5-4442-4e0a-8f02-cdd0ce04bf0a None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:20.041036 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ad9c5-4442-4e0a-8f02-cdd0ce04bf0a None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:20.041114 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ad9c5-4442-4e0a-8f02-cdd0ce04bf0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:20.072211 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-ae4ad9c5-4442-4e0a-8f02-cdd0ce04bf0a None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:01:20.073408 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-ae4ad9c5-4442-4e0a-8f02-cdd0ce04bf0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:01:20.212705 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 230/457] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:01:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:01:20.268972 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c77789b-a506-4b35-89a1-4edee0f564a4 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:20.269249 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c77789b-a506-4b35-89a1-4edee0f564a4 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:20.269708 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c77789b-a506-4b35-89a1-4edee0f564a4 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:20.271178 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 228/458] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:01:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:01:20.276905 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e820a1ef-5fa0-437e-ae30-8c932fab53a9 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:20.277454 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e820a1ef-5fa0-437e-ae30-8c932fab53a9 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:20.277920 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e820a1ef-5fa0-437e-ae30-8c932fab53a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:20.315337 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de769edf-cdc0-4b44-8517-99ad351a26a1 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:20.315337 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de769edf-cdc0-4b44-8517-99ad351a26a1 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:20.315337 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de769edf-cdc0-4b44-8517-99ad351a26a1 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:20.315629 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-e820a1ef-5fa0-437e-ae30-8c932fab53a9 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:01:20.315926 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-e820a1ef-5fa0-437e-ae30-8c932fab53a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:01:20.316858 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 229/459] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:01:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:01:20.322280 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4905112-3c7c-409a-af2d-b7e22b83c7ac None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:20.322781 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4905112-3c7c-409a-af2d-b7e22b83c7ac None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:20.323217 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4905112-3c7c-409a-af2d-b7e22b83c7ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:20.358910 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-f4905112-3c7c-409a-af2d-b7e22b83c7ac None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:01:20.362397 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-f4905112-3c7c-409a-af2d-b7e22b83c7ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:01:20.403351 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:20.444475 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 231/460] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:01:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 12:01:20.487326 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 230/461] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:01:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 12:01:20.657060 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e820a1ef-5fa0-437e-ae30-8c932fab53a9 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:20.789987 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e820a1ef-5fa0-437e-ae30-8c932fab53a9 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:20.806856 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4905112-3c7c-409a-af2d-b7e22b83c7ac None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:20.831764 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:20.836470 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f250a3-6ffe-452f-9081-104451c52d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:20.837097 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f250a3-6ffe-452f-9081-104451c52d52 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:20.837368 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f250a3-6ffe-452f-9081-104451c52d52 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:20.837795 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f250a3-6ffe-452f-9081-104451c52d52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:20.849440 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f250a3-6ffe-452f-9081-104451c52d52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:20.851091 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f250a3-6ffe-452f-9081-104451c52d52 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:20.868201 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-f3f250a3-6ffe-452f-9081-104451c52d52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:20.868201 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:20.868201 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:20.868201 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:20.868201 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:20.868201 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:20.868201 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:20.868201 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:20.868201 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:20.868201 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:20.868201 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:20.868201 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:20.868201 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:01:20.868201 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:01:20.868201 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:01:20.868201 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:01:20.869150 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:20.869150 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:20.869150 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:20.869150 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:20.869150 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:20.869150 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:20.869150 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:20.869150 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:20.869150 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:20.869150 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:20.869150 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:20.869150 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:20.869150 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:20.869150 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:20.869150 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:20.869150 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:20.869150 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:01:20.870018 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:01:20.870018 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:20.870018 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:20.870018 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 232/462] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:01:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:20.870778 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287c4172-4ff2-4194-9bdf-1ec6ef4da0ee None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:20.870778 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287c4172-4ff2-4194-9bdf-1ec6ef4da0ee None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:20.871008 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287c4172-4ff2-4194-9bdf-1ec6ef4da0ee None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:20.872481 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 233/463] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:01:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:01:20.873927 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287c4172-4ff2-4194-9bdf-1ec6ef4da0ee None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:20.907298 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:20.937411 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4905112-3c7c-409a-af2d-b7e22b83c7ac None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:20.971713 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5f8d76-903e-449c-a86d-f2db101c7082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:20.972345 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f8d76-903e-449c-a86d-f2db101c7082 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:20.972628 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f8d76-903e-449c-a86d-f2db101c7082 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:20.972906 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f8d76-903e-449c-a86d-f2db101c7082 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:20.983876 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5f8d76-903e-449c-a86d-f2db101c7082 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:20.985320 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5f8d76-903e-449c-a86d-f2db101c7082 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:20.987436 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287c4172-4ff2-4194-9bdf-1ec6ef4da0ee None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:21.000837 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-ad5f8d76-903e-449c-a86d-f2db101c7082 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:21.000837 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:21.000837 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:21.000837 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:21.000837 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:21.000837 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:21.000837 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:21.000837 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:21.000837 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:21.000837 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:21.000837 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:21.000837 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:21.000837 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:01:21.000837 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:01:21.000837 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:01:21.000837 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:01:21.001758 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:21.001758 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:21.001758 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:21.001758 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:21.001758 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:21.001758 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:21.001758 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:21.001758 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:21.001758 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:21.001758 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:21.001758 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:21.001758 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:21.001758 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:21.001758 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:21.001758 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:21.001758 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:21.001758 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:01:21.002617 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:01:21.002617 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:21.002617 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:21.002617 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 231/464] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:01:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 12:01:21.003192 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5f8d76-903e-449c-a86d-f2db101c7082 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:21.030666 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9206e00c-8d12-4735-b782-4e410b7ab823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:21.031325 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206e00c-8d12-4735-b782-4e410b7ab823 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:21.031609 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206e00c-8d12-4735-b782-4e410b7ab823 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:21.031877 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206e00c-8d12-4735-b782-4e410b7ab823 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:21.043733 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9206e00c-8d12-4735-b782-4e410b7ab823 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:21.045801 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9206e00c-8d12-4735-b782-4e410b7ab823 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:21.061390 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-9206e00c-8d12-4735-b782-4e410b7ab823 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:21.061390 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:21.061390 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:21.061390 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:21.061390 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:21.061390 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:21.061390 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:21.061390 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:21.061390 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:21.061390 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:21.061390 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:21.061390 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:21.061390 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:01:21.061390 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:01:21.061390 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:01:21.061390 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:01:21.062444 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:21.062444 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:21.062444 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:21.062444 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:21.062444 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:21.062444 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:21.062444 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:21.062444 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:21.062444 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:21.062444 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:21.062444 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:21.062444 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:21.062444 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:21.062444 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:21.062444 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:21.062444 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:21.062444 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:01:21.063347 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:01:21.063347 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:21.063347 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:21.063347 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 234/465] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:01:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:21.063770 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b47b2-7309-42d2-85d7-de9734124465 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:21.064057 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b47b2-7309-42d2-85d7-de9734124465 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:21.064328 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b47b2-7309-42d2-85d7-de9734124465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:21.083857 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-ec6b47b2-7309-42d2-85d7-de9734124465 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:01:21.084782 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:21.085221 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-ec6b47b2-7309-42d2-85d7-de9734124465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:01:21.120388 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5f8d76-903e-449c-a86d-f2db101c7082 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:21.154622 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 235/466] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:01:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:01:21.155740 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6b47b2-7309-42d2-85d7-de9734124465 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:21.165048 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60efeed7-8239-436c-bafc-6ea0e7233a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:21.166412 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60efeed7-8239-436c-bafc-6ea0e7233a73 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:21.166948 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60efeed7-8239-436c-bafc-6ea0e7233a73 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:21.167823 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60efeed7-8239-436c-bafc-6ea0e7233a73 None admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:21.168973 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60efeed7-8239-436c-bafc-6ea0e7233a73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:21.170061 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60efeed7-8239-436c-bafc-6ea0e7233a73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:21.172203 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:21.174060 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60efeed7-8239-436c-bafc-6ea0e7233a73 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:21.196542 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 232/467] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:01:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:21.196542 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60efeed7-8239-436c-bafc-6ea0e7233a73 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:21.279850 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6b47b2-7309-42d2-85d7-de9734124465 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:21.304902 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60efeed7-8239-436c-bafc-6ea0e7233a73 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:21.318149 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39b463f-af65-48b6-87c8-9033bb11578a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:21.319123 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b463f-af65-48b6-87c8-9033bb11578a None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:21.319848 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b463f-af65-48b6-87c8-9033bb11578a None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:21.320315 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b463f-af65-48b6-87c8-9033bb11578a None admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:21.321186 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39b463f-af65-48b6-87c8-9033bb11578a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:21.321717 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39b463f-af65-48b6-87c8-9033bb11578a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:21.323496 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39b463f-af65-48b6-87c8-9033bb11578a None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:21.335172 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:21.337712 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 236/468] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:01:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:21.339083 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39b463f-af65-48b6-87c8-9033bb11578a None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:21.347723 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306beed7-ed84-42f5-89af-5e629e7099cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:21.348746 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306beed7-ed84-42f5-89af-5e629e7099cf None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:21.349222 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306beed7-ed84-42f5-89af-5e629e7099cf None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:21.349680 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306beed7-ed84-42f5-89af-5e629e7099cf None admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:21.350680 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306beed7-ed84-42f5-89af-5e629e7099cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:21.351025 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306beed7-ed84-42f5-89af-5e629e7099cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:21.352572 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306beed7-ed84-42f5-89af-5e629e7099cf None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:21.365867 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 233/469] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:01:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:21.366842 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306beed7-ed84-42f5-89af-5e629e7099cf None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:21.426183 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:21.454844 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39b463f-af65-48b6-87c8-9033bb11578a None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:21.483583 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306beed7-ed84-42f5-89af-5e629e7099cf None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:21.497163 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffe9578-5a96-4208-872b-b18df1caf74c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:21.498368 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe9578-5a96-4208-872b-b18df1caf74c None admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:21.499188 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe9578-5a96-4208-872b-b18df1caf74c None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:21.499966 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe9578-5a96-4208-872b-b18df1caf74c None admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:21.500805 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffe9578-5a96-4208-872b-b18df1caf74c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:21.502798 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffe9578-5a96-4208-872b-b18df1caf74c None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:21.521152 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f3315d-d180-43a1-808d-4ae5712dbfab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:21.521781 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f3315d-d180-43a1-808d-4ae5712dbfab None admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:21.522075 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f3315d-d180-43a1-808d-4ae5712dbfab None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:21.522402 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f3315d-d180-43a1-808d-4ae5712dbfab None admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:21.523238 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f3315d-d180-43a1-808d-4ae5712dbfab None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:21.524690 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f3315d-d180-43a1-808d-4ae5712dbfab None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:21.529434 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 237/470] 10.222.0.61 () {60 vars in 1132 bytes} [Fri Aug 26 12:01:21 2022] POST /identity/v3/users => generated 246 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:01:21.530481 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffe9578-5a96-4208-872b-b18df1caf74c None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:21.557288 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 234/471] 10.222.0.61 () {60 vars in 1132 bytes} [Fri Aug 26 12:01:21 2022] POST /identity/v3/users => generated 248 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:01:21.558366 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f3315d-d180-43a1-808d-4ae5712dbfab None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:21.629115 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:21.697247 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f3315d-d180-43a1-808d-4ae5712dbfab None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:21.728417 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffe9578-5a96-4208-872b-b18df1caf74c None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:21.742868 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9630ad85-bda2-4127-808e-ef358740c5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:21.743555 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9630ad85-bda2-4127-808e-ef358740c5b7 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:21.743838 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9630ad85-bda2-4127-808e-ef358740c5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:21.744114 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9630ad85-bda2-4127-808e-ef358740c5b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:21.763631 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9630ad85-bda2-4127-808e-ef358740c5b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:21.765405 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9630ad85-bda2-4127-808e-ef358740c5b7 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:21.769531 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:21.776906 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f3bec2-6b7d-4487-9315-64556070565f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:21.777562 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f3bec2-6b7d-4487-9315-64556070565f None admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:21.778096 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f3bec2-6b7d-4487-9315-64556070565f None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:21.778381 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f3bec2-6b7d-4487-9315-64556070565f None admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:21.779581 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f3bec2-6b7d-4487-9315-64556070565f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:21.781054 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f3bec2-6b7d-4487-9315-64556070565f None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:21.783782 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-9630ad85-bda2-4127-808e-ef358740c5b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:21.783782 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:21.783782 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:21.783782 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:21.783782 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:21.783782 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:21.783782 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:21.783782 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:21.783782 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:21.783782 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:21.783782 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:21.783782 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:21.783782 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:01:21.783782 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:01:21.783782 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:01:21.783782 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:01:21.784780 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:21.784780 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:21.784780 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:21.784780 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:21.784780 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:21.784780 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:21.784780 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:21.784780 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:21.784780 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:21.784780 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:21.784780 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:21.784780 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:21.784780 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:21.784780 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:21.784780 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:21.784780 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:21.784780 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:01:21.785622 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:01:21.785622 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:21.785622 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:21.787052 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 235/472] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:01:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:21.791191 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9630ad85-bda2-4127-808e-ef358740c5b7 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:21.809265 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 238/473] 10.222.0.61 () {60 vars in 1132 bytes} [Fri Aug 26 12:01:21 2022] POST /identity/v3/users => generated 248 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:01:21.830418 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9516f9e7-4a1d-40e1-99f2-9c30499167d5 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:21.830711 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9516f9e7-4a1d-40e1-99f2-9c30499167d5 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:21.830976 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9516f9e7-4a1d-40e1-99f2-9c30499167d5 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:21.833447 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 239/474] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:01:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:01:21.839029 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464d718a-f889-4ab3-b1bb-2247a5563337 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:21.840246 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464d718a-f889-4ab3-b1bb-2247a5563337 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:21.840531 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464d718a-f889-4ab3-b1bb-2247a5563337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:21.863650 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-464d718a-f889-4ab3-b1bb-2247a5563337 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:01:21.865058 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-464d718a-f889-4ab3-b1bb-2247a5563337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:01:21.925533 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:21.929489 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9630ad85-bda2-4127-808e-ef358740c5b7 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:21.970786 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 240/475] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:01:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 12:01:21.973927 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f11195-3769-4033-bc96-f59884ed384d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:21.974618 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f11195-3769-4033-bc96-f59884ed384d None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:21.974963 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f11195-3769-4033-bc96-f59884ed384d None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:21.975335 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f11195-3769-4033-bc96-f59884ed384d None admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:21.976067 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f11195-3769-4033-bc96-f59884ed384d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:21.976440 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f11195-3769-4033-bc96-f59884ed384d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:21.977948 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f11195-3769-4033-bc96-f59884ed384d None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:21.992774 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 236/476] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:01:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:01:21.999230 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464d718a-f889-4ab3-b1bb-2247a5563337 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:22.104016 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464d718a-f889-4ab3-b1bb-2247a5563337 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:22.123463 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:22.144487 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceff6ce-1409-4470-bc1c-7095f68a4398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:22.145095 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceff6ce-1409-4470-bc1c-7095f68a4398 None admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:22.145371 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceff6ce-1409-4470-bc1c-7095f68a4398 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:22.145641 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceff6ce-1409-4470-bc1c-7095f68a4398 None admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:22.146278 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceff6ce-1409-4470-bc1c-7095f68a4398 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:22.147760 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceff6ce-1409-4470-bc1c-7095f68a4398 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:22.172833 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 241/477] 10.222.0.61 () {60 vars in 1132 bytes} [Fri Aug 26 12:01:21 2022] POST /identity/v3/users => generated 249 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:01:22.272405 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f11195-3769-4033-bc96-f59884ed384d None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:22.386856 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f11195-3769-4033-bc96-f59884ed384d None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:22.423862 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313ae666-8a7a-437f-86ef-87c0c956e38c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:22.424530 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ae666-8a7a-437f-86ef-87c0c956e38c None admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:22.424805 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ae666-8a7a-437f-86ef-87c0c956e38c None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:22.425074 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ae666-8a7a-437f-86ef-87c0c956e38c None admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:22.425747 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313ae666-8a7a-437f-86ef-87c0c956e38c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:22.427129 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313ae666-8a7a-437f-86ef-87c0c956e38c None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:22.436589 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.sql.core [None req-313ae666-8a7a-437f-86ef-87c0c956e38c None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 26 12:01:22.436589 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 26 12:01:22.436589 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [parameters: {'id': '9fee8aaf98424b22af27cdaf012a03ac', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 26 12:01:22.436589 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81712) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 26 12:01:22.453885 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-313ae666-8a7a-437f-86ef-87c0c956e38c None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 26 12:01:22.453885 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:22.453885 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 26 12:01:22.453885 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 26 12:01:22.453885 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 26 12:01:22.453885 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 26 12:01:22.453885 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 26 12:01:22.453885 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application result = self._query(query) Aug 26 12:01:22.453885 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 26 12:01:22.453885 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application conn.query(q) Aug 26 12:01:22.453885 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 26 12:01:22.453885 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 26 12:01:22.453885 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 26 12:01:22.453885 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application result.read() Aug 26 12:01:22.453885 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 26 12:01:22.455893 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 26 12:01:22.455893 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 26 12:01:22.455893 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 26 12:01:22.455893 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 26 12:01:22.455893 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 26 12:01:22.455893 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 26 12:01:22.455893 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 26 12:01:22.455893 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 26 12:01:22.455893 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:22.455893 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 26 12:01:22.455893 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:22.455893 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:22.455893 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 26 12:01:22.455893 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 26 12:01:22.455893 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 26 12:01:22.455893 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return ref.to_dict() Aug 26 12:01:22.455893 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 26 12:01:22.455893 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application next(self.gen) Aug 26 12:01:22.458047 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 26 12:01:22.458047 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application yield resource Aug 26 12:01:22.458047 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 26 12:01:22.458047 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application next(self.gen) Aug 26 12:01:22.458047 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 26 12:01:22.458047 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application self.session.rollback() Aug 26 12:01:22.458047 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 26 12:01:22.458047 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application self.force_reraise() Aug 26 12:01:22.458047 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 26 12:01:22.458047 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise self.value Aug 26 12:01:22.458047 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 26 12:01:22.458047 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 26 12:01:22.458047 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 26 12:01:22.458047 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application session.commit() Aug 26 12:01:22.458047 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 26 12:01:22.458047 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 26 12:01:22.458047 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 26 12:01:22.460080 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application self._prepare_impl() Aug 26 12:01:22.460080 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 26 12:01:22.460080 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application self.session.flush() Aug 26 12:01:22.460080 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 26 12:01:22.460080 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application self._flush(objects) Aug 26 12:01:22.460080 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 26 12:01:22.460080 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 26 12:01:22.460080 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 26 12:01:22.460080 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application compat.raise_( Aug 26 12:01:22.460080 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 26 12:01:22.460080 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception Aug 26 12:01:22.460080 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 26 12:01:22.460080 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application flush_context.execute() Aug 26 12:01:22.460080 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 26 12:01:22.460080 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rec.execute(self) Aug 26 12:01:22.460080 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 26 12:01:22.460080 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 26 12:01:22.462108 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 26 12:01:22.462108 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 26 12:01:22.462108 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 26 12:01:22.462108 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 26 12:01:22.462108 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 26 12:01:22.462108 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 26 12:01:22.462108 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 26 12:01:22.462108 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 26 12:01:22.462108 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 26 12:01:22.462108 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 26 12:01:22.462108 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 26 12:01:22.462108 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 26 12:01:22.462108 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 26 12:01:22.462108 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 26 12:01:22.462108 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 26 12:01:22.462108 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception Aug 26 12:01:22.465349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 26 12:01:22.465349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 26 12:01:22.465349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 26 12:01:22.465349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 26 12:01:22.465349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 26 12:01:22.465349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application result = self._query(query) Aug 26 12:01:22.465349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 26 12:01:22.465349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application conn.query(q) Aug 26 12:01:22.465349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 26 12:01:22.465349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 26 12:01:22.465349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 26 12:01:22.465349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application result.read() Aug 26 12:01:22.465349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 26 12:01:22.465349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 26 12:01:22.465349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 26 12:01:22.465349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 26 12:01:22.468032 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 26 12:01:22.468032 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 26 12:01:22.468032 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 26 12:01:22.468032 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 26 12:01:22.468032 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 26 12:01:22.468032 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 26 12:01:22.468032 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [parameters: {'id': '9fee8aaf98424b22af27cdaf012a03ac', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 26 12:01:22.468032 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 26 12:01:22.468032 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:22.468032 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 12:01:22.468032 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:22.468032 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:22.468032 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:22.468032 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:22.468032 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:22.468032 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:22.469909 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:22.469909 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:22.469909 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:22.469909 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:22.469909 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:22.469909 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:22.469909 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 26 12:01:22.469909 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 26 12:01:22.469909 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:22.469909 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:22.469909 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 26 12:01:22.469909 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 26 12:01:22.469909 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 26 12:01:22.469909 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 26 12:01:22.469909 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 26 12:01:22.469909 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:22.472616 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 237/478] 10.222.0.61 () {60 vars in 1131 bytes} [Fri Aug 26 12:01:22 2022] POST /identity/v3/roles => generated 144 bytes in 184 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:22.472761 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceff6ce-1409-4470-bc1c-7095f68a4398 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:22.594850 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceff6ce-1409-4470-bc1c-7095f68a4398 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:22.630632 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bc4b4c-5a34-4a1e-b9ba-1327aa9074b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:22.631293 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bc4b4c-5a34-4a1e-b9ba-1327aa9074b4 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:22.631573 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bc4b4c-5a34-4a1e-b9ba-1327aa9074b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:22.631838 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bc4b4c-5a34-4a1e-b9ba-1327aa9074b4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:22.643222 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bc4b4c-5a34-4a1e-b9ba-1327aa9074b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:01:22.648455 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-f9bc4b4c-5a34-4a1e-b9ba-1327aa9074b4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 12:01:22.648455 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:22.648455 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:22.648455 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:22.648455 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:22.648455 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:22.648455 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:22.648455 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:22.648455 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:22.648455 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:22.648455 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:22.648455 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:22.648455 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:01:22.648455 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:01:22.648455 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:01:22.648455 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:01:22.649400 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:01:22.649400 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:01:22.649400 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:01:22.649400 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:01:22.649400 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:22.649400 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:22.649400 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:22.649400 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:22.649400 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:22.649400 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:22.649400 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:22.649400 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:22.649400 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:22.649400 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:22.649400 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:22.649400 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:22.650191 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:22.650191 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:22.650191 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:22.650191 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:22.650191 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:01:22.650191 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:01:22.650191 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:01:22.650191 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:01:22.650191 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:01:22.650191 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:01:22.650191 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 26 12:01:22.650191 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:22.650191 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 242/479] 10.222.0.61 () {56 vars in 1114 bytes} [Fri Aug 26 12:01:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:22.656017 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313ae666-8a7a-437f-86ef-87c0c956e38c None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:22.692876 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:22.803568 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313ae666-8a7a-437f-86ef-87c0c956e38c None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:22.851872 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0920d485-b5ab-401a-9351-9d4c13595863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:22.852398 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0920d485-b5ab-401a-9351-9d4c13595863 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:22.852674 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0920d485-b5ab-401a-9351-9d4c13595863 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:22.852943 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0920d485-b5ab-401a-9351-9d4c13595863 None admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:22.853607 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0920d485-b5ab-401a-9351-9d4c13595863 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:22.853908 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0920d485-b5ab-401a-9351-9d4c13595863 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:22.855299 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0920d485-b5ab-401a-9351-9d4c13595863 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:22.866645 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 238/480] 10.222.0.61 () {56 vars in 1111 bytes} [Fri Aug 26 12:01:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:23.859240 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c60e50-a087-4524-a47a-d56d4573cf7f None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:23.860245 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c60e50-a087-4524-a47a-d56d4573cf7f None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:23.860754 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c60e50-a087-4524-a47a-d56d4573cf7f None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:23.862748 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 243/481] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:01:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:01:23.868380 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d044e-5c53-410b-9448-cd9a7eb671a9 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:23.869176 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d044e-5c53-410b-9448-cd9a7eb671a9 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:23.869645 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d044e-5c53-410b-9448-cd9a7eb671a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:23.889195 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-b95d044e-5c53-410b-9448-cd9a7eb671a9 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:01:23.890817 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-b95d044e-5c53-410b-9448-cd9a7eb671a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:01:23.966827 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 239/482] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:01:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 12:01:24.194368 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d7b48-7c7c-4258-8bcf-86bbe662a41e None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:24.195048 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d7b48-7c7c-4258-8bcf-86bbe662a41e None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:24.195583 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d7b48-7c7c-4258-8bcf-86bbe662a41e None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:24.197382 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 244/483] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:01:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:01:24.203710 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f66d5d7-fc06-4767-b683-dd4279a0e00c None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:24.204215 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f66d5d7-fc06-4767-b683-dd4279a0e00c None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:24.204684 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f66d5d7-fc06-4767-b683-dd4279a0e00c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:24.225762 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-4f66d5d7-fc06-4767-b683-dd4279a0e00c None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:01:24.227324 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-4f66d5d7-fc06-4767-b683-dd4279a0e00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:01:24.241537 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:24.305561 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849d7b48-7c7c-4258-8bcf-86bbe662a41e None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:24.324155 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 240/484] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:01:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:01:24.325459 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e189632-4793-4c28-aa76-b6d12d964ce6 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:24.326329 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e189632-4793-4c28-aa76-b6d12d964ce6 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:24.326604 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e189632-4793-4c28-aa76-b6d12d964ce6 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:24.327906 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 241/485] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:01:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:01:24.333111 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1124e-37d1-4837-9a25-1a40ed4db3cc None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:24.333396 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1124e-37d1-4837-9a25-1a40ed4db3cc None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:24.333678 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1124e-37d1-4837-9a25-1a40ed4db3cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:24.342812 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:24.352427 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-f7f1124e-37d1-4837-9a25-1a40ed4db3cc None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:01:24.353600 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-f7f1124e-37d1-4837-9a25-1a40ed4db3cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:01:24.425417 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 242/486] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:01:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 12:01:24.483968 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849d7b48-7c7c-4258-8bcf-86bbe662a41e None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:24.531553 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea20c8aa-9078-459e-ab5a-44a22d0db4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:24.532244 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea20c8aa-9078-459e-ab5a-44a22d0db4d1 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:24.532530 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea20c8aa-9078-459e-ab5a-44a22d0db4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:24.532803 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea20c8aa-9078-459e-ab5a-44a22d0db4d1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:24.551366 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea20c8aa-9078-459e-ab5a-44a22d0db4d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:01:24.556115 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-ea20c8aa-9078-459e-ab5a-44a22d0db4d1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 12:01:24.556115 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:24.556115 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:24.556115 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:24.556115 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:24.556115 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:24.556115 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:24.556115 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:24.556115 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:24.556115 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:24.556115 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:24.556115 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:24.556115 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:01:24.556115 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:01:24.556115 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:01:24.556115 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:01:24.557003 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:01:24.557003 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:01:24.557003 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:01:24.557003 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:01:24.557003 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:24.557003 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:24.557003 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:24.557003 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:24.557003 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:24.557003 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:24.557003 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:24.557003 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:24.557003 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:24.557003 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:24.557003 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:24.557003 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:24.557828 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:24.557828 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:24.557828 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:24.557828 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:24.557828 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:01:24.557828 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:01:24.557828 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:01:24.557828 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:01:24.557828 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:01:24.557828 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:01:24.557828 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 12:01:24.557828 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:24.560554 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 245/487] 10.222.0.61 () {56 vars in 1102 bytes} [Fri Aug 26 12:01:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:24.568299 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f1124e-37d1-4837-9a25-1a40ed4db3cc None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:24.668407 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea20c8aa-9078-459e-ab5a-44a22d0db4d1 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:24.673951 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:24.701663 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f1124e-37d1-4837-9a25-1a40ed4db3cc None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:24.753965 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebff2bc-c77f-4c00-8de5-7d738fad44de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:24.754974 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebff2bc-c77f-4c00-8de5-7d738fad44de None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:24.755501 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebff2bc-c77f-4c00-8de5-7d738fad44de None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:24.755968 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebff2bc-c77f-4c00-8de5-7d738fad44de None admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:24.756841 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebff2bc-c77f-4c00-8de5-7d738fad44de None admin] RBAC: Adding query filter params (name=service) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:24.757446 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebff2bc-c77f-4c00-8de5-7d738fad44de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:24.759312 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebff2bc-c77f-4c00-8de5-7d738fad44de None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:24.769860 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 243/488] 10.222.0.61 () {56 vars in 1103 bytes} [Fri Aug 26 12:01:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:24.771447 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c8c21-2c06-48e5-8702-cca5390e5e3d None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:24.771929 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c8c21-2c06-48e5-8702-cca5390e5e3d None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:24.772442 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c8c21-2c06-48e5-8702-cca5390e5e3d None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:24.774598 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 244/489] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:01:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:01:24.775554 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1c8c21-2c06-48e5-8702-cca5390e5e3d None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:24.807153 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:24.818584 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea20c8aa-9078-459e-ab5a-44a22d0db4d1 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:24.854626 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a12997-a672-4f79-b776-e6bd88845461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:24.855283 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a12997-a672-4f79-b776-e6bd88845461 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:24.855563 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a12997-a672-4f79-b776-e6bd88845461 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:24.855837 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a12997-a672-4f79-b776-e6bd88845461 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:24.866023 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a12997-a672-4f79-b776-e6bd88845461 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:01:24.871220 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-a7a12997-a672-4f79-b776-e6bd88845461 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 12:01:24.871220 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:24.871220 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:24.871220 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:24.871220 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:24.871220 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:24.871220 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:24.871220 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:24.871220 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:24.871220 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:24.871220 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:24.871220 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:24.871220 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:01:24.871220 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:01:24.871220 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:01:24.871220 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:01:24.872114 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:01:24.872114 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:01:24.872114 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:01:24.872114 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:01:24.872114 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:24.872114 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:24.872114 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:24.872114 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:24.872114 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:24.872114 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:24.872114 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:24.872114 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:24.872114 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:24.872114 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:24.872114 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:24.872114 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:24.873526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:24.873526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:24.873526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:24.873526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:24.873526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:01:24.873526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:01:24.873526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:01:24.873526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:01:24.873526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:01:24.873526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:01:24.873526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 12:01:24.873526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:24.873526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 246/490] 10.222.0.61 () {56 vars in 1102 bytes} [Fri Aug 26 12:01:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 12:01:24.874208 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2317c36-aadc-4445-a280-979f1e57d27d None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:24.874609 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2317c36-aadc-4445-a280-979f1e57d27d None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:24.874609 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2317c36-aadc-4445-a280-979f1e57d27d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:24.894166 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-b2317c36-aadc-4445-a280-979f1e57d27d None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:01:24.895376 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-b2317c36-aadc-4445-a280-979f1e57d27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:01:24.897026 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1c8c21-2c06-48e5-8702-cca5390e5e3d None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:24.933935 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648191a4-3dac-4cde-a3c0-299662d8b1da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:24.934556 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648191a4-3dac-4cde-a3c0-299662d8b1da None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:24.935484 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648191a4-3dac-4cde-a3c0-299662d8b1da None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:24.935761 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648191a4-3dac-4cde-a3c0-299662d8b1da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:24.944017 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:24.948385 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648191a4-3dac-4cde-a3c0-299662d8b1da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:24.949922 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648191a4-3dac-4cde-a3c0-299662d8b1da None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:24.967320 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-648191a4-3dac-4cde-a3c0-299662d8b1da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:24.967320 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:24.967320 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:24.967320 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:24.967320 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:24.967320 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:24.967320 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:24.967320 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:24.967320 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:24.967320 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:24.967320 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:24.967320 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:24.967320 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:01:24.967320 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:01:24.967320 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:01:24.967320 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:01:24.968307 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:24.968307 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:24.968307 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:24.968307 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:24.968307 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:24.968307 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:24.968307 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:24.968307 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:24.968307 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:24.968307 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:24.968307 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:24.968307 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:24.968307 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:24.968307 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:24.968307 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:24.968307 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:24.968307 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:01:24.969229 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:01:24.969229 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:24.969229 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:24.969229 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 245/491] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:01:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:24.969875 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648191a4-3dac-4cde-a3c0-299662d8b1da None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:24.971804 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 247/492] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:01:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:01:24.972644 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2317c36-aadc-4445-a280-979f1e57d27d None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:25.060508 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:25.080046 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648191a4-3dac-4cde-a3c0-299662d8b1da None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:25.080981 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2317c36-aadc-4445-a280-979f1e57d27d None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:25.115064 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b809e5-7dbb-4251-8639-e50e8a5a2b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:25.116064 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b809e5-7dbb-4251-8639-e50e8a5a2b7d None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:25.116674 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b809e5-7dbb-4251-8639-e50e8a5a2b7d None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:25.117141 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b809e5-7dbb-4251-8639-e50e8a5a2b7d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:25.120023 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0620e423-581b-448f-9d8f-4e0c8e1fde6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:25.120558 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0620e423-581b-448f-9d8f-4e0c8e1fde6d None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:25.120845 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0620e423-581b-448f-9d8f-4e0c8e1fde6d None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:25.121167 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0620e423-581b-448f-9d8f-4e0c8e1fde6d None admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:25.121827 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0620e423-581b-448f-9d8f-4e0c8e1fde6d None admin] RBAC: Adding query filter params (name=service) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:25.122131 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0620e423-581b-448f-9d8f-4e0c8e1fde6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:25.123613 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0620e423-581b-448f-9d8f-4e0c8e1fde6d None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:25.129356 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b809e5-7dbb-4251-8639-e50e8a5a2b7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:01:25.132853 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 248/493] 10.222.0.61 () {56 vars in 1103 bytes} [Fri Aug 26 12:01:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:25.133915 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0620e423-581b-448f-9d8f-4e0c8e1fde6d None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:25.135852 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-a4b809e5-7dbb-4251-8639-e50e8a5a2b7d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 12:01:25.135852 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:25.135852 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:25.135852 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:25.135852 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:25.135852 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:25.135852 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:25.135852 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:25.135852 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:25.135852 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:25.135852 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:25.135852 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:25.135852 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:01:25.135852 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:01:25.135852 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:01:25.135852 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:01:25.136753 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:01:25.136753 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:01:25.136753 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:01:25.136753 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:01:25.136753 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:25.136753 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:25.136753 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:25.136753 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:25.136753 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:25.136753 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:25.136753 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:25.136753 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:25.136753 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:25.136753 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:25.136753 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:25.136753 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:25.137562 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:25.137562 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:25.137562 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:25.137562 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:25.137562 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:01:25.137562 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:01:25.137562 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:01:25.137562 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:01:25.137562 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:01:25.137562 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:01:25.137562 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 12:01:25.137562 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:25.145353 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 246/494] 10.222.0.61 () {56 vars in 1102 bytes} [Fri Aug 26 12:01:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:25.146441 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b809e5-7dbb-4251-8639-e50e8a5a2b7d None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:25.197408 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:25.246607 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0620e423-581b-448f-9d8f-4e0c8e1fde6d None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:25.250659 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b809e5-7dbb-4251-8639-e50e8a5a2b7d None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:25.284720 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8c170e-581a-4673-95c1-bb1c979368ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:25.285324 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8c170e-581a-4673-95c1-bb1c979368ca None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:25.285608 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8c170e-581a-4673-95c1-bb1c979368ca None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:25.285885 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8c170e-581a-4673-95c1-bb1c979368ca None admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:25.286527 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8c170e-581a-4673-95c1-bb1c979368ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:25.286832 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8c170e-581a-4673-95c1-bb1c979368ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:25.288024 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2821565-b48d-482a-88d7-0f013bbb17ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:25.288239 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8c170e-581a-4673-95c1-bb1c979368ca None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:25.288650 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2821565-b48d-482a-88d7-0f013bbb17ea None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:25.288928 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2821565-b48d-482a-88d7-0f013bbb17ea None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:25.289196 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2821565-b48d-482a-88d7-0f013bbb17ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:25.300163 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2821565-b48d-482a-88d7-0f013bbb17ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:25.300488 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 249/495] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:01:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:25.301290 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8c170e-581a-4673-95c1-bb1c979368ca None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:25.301618 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2821565-b48d-482a-88d7-0f013bbb17ea None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:25.305659 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:25.316667 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-e2821565-b48d-482a-88d7-0f013bbb17ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:25.316667 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:25.316667 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:25.316667 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:25.316667 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:25.316667 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:25.316667 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:25.316667 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:25.316667 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:25.316667 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:25.316667 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:25.316667 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:25.316667 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:01:25.316667 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:01:25.316667 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:01:25.316667 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:01:25.317565 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:25.317565 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:25.317565 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:25.317565 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:25.317565 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:25.317565 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:25.317565 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:25.317565 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:25.317565 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:25.317565 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:25.317565 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:25.317565 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:25.317565 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:25.317565 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:25.317565 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:25.317565 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:25.317565 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:01:25.318400 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:01:25.318400 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:25.318400 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:25.322467 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 247/496] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:01:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:25.323322 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2821565-b48d-482a-88d7-0f013bbb17ea None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:25.414700 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8c170e-581a-4673-95c1-bb1c979368ca None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:25.438238 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2821565-b48d-482a-88d7-0f013bbb17ea None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:25.439797 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:25.455458 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5638c99c-0ff3-4dca-b6a0-26fe66a1237d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:25.456166 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5638c99c-0ff3-4dca-b6a0-26fe66a1237d None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:25.456472 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5638c99c-0ff3-4dca-b6a0-26fe66a1237d None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:25.456830 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5638c99c-0ff3-4dca-b6a0-26fe66a1237d None admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:25.457480 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5638c99c-0ff3-4dca-b6a0-26fe66a1237d None admin] RBAC: Adding query filter params (name=service) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:25.457776 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5638c99c-0ff3-4dca-b6a0-26fe66a1237d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:25.459128 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5638c99c-0ff3-4dca-b6a0-26fe66a1237d None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:25.473056 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 250/497] 10.222.0.61 () {56 vars in 1103 bytes} [Fri Aug 26 12:01:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:25.473056 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5638c99c-0ff3-4dca-b6a0-26fe66a1237d None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:25.484141 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c52a11-9c6e-4156-b560-45485e087444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:25.484782 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c52a11-9c6e-4156-b560-45485e087444 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:25.485068 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c52a11-9c6e-4156-b560-45485e087444 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:25.485344 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c52a11-9c6e-4156-b560-45485e087444 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:25.495712 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c52a11-9c6e-4156-b560-45485e087444 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:01:25.502699 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-b0c52a11-9c6e-4156-b560-45485e087444 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 12:01:25.502699 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:25.502699 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:25.502699 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:25.502699 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:25.502699 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:25.502699 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:25.502699 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:25.502699 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:25.502699 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:25.502699 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:25.502699 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:25.502699 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:01:25.502699 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:01:25.502699 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:01:25.502699 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:01:25.511045 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:01:25.511045 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:01:25.511045 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:01:25.511045 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:01:25.511045 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:25.511045 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:25.511045 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:25.511045 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:25.511045 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:25.511045 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:25.511045 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:25.511045 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:25.511045 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:25.511045 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:25.511045 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:25.511045 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:25.512041 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:25.512041 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:25.512041 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:25.512041 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:25.512041 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:01:25.512041 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:01:25.512041 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:01:25.512041 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:01:25.512041 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:01:25.512041 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:01:25.512041 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 12:01:25.512041 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:25.512041 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 248/498] 10.222.0.61 () {56 vars in 1102 bytes} [Fri Aug 26 12:01:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:25.512041 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c52a11-9c6e-4156-b560-45485e087444 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:25.553924 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:25.595502 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5638c99c-0ff3-4dca-b6a0-26fe66a1237d None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:25.617422 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c52a11-9c6e-4156-b560-45485e087444 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:25.631726 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15efe227-39ba-4ae1-9f27-51f5c718bcfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:25.632346 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15efe227-39ba-4ae1-9f27-51f5c718bcfb None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:25.632626 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15efe227-39ba-4ae1-9f27-51f5c718bcfb None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:25.632901 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15efe227-39ba-4ae1-9f27-51f5c718bcfb None admin] PATH_INFO: `/v3/users/nova` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:25.641279 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15efe227-39ba-4ae1-9f27-51f5c718bcfb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:25.642717 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15efe227-39ba-4ae1-9f27-51f5c718bcfb None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:25.655767 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b554de-8863-4826-9810-4c05e80760e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:25.655928 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-15efe227-39ba-4ae1-9f27-51f5c718bcfb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 12:01:25.655928 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:25.655928 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:25.655928 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:25.655928 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:25.655928 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:25.655928 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:25.655928 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:25.655928 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:25.655928 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:25.655928 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:25.655928 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:25.655928 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 12:01:25.655928 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 12:01:25.655928 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 12:01:25.655928 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 12:01:25.656768 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:25.656768 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:25.656768 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 12:01:25.656768 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:01:25.656768 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 12:01:25.656768 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:01:25.656768 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:25.656768 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:25.656768 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:25.656768 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:25.656768 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:25.656768 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:25.656768 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:25.656768 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:25.656768 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:25.656768 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:25.656768 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:25.656768 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:25.657738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:25.657738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:25.657738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 12:01:25.657738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 12:01:25.657738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 12:01:25.657738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 12:01:25.657738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 12:01:25.657738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 12:01:25.657738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 12:01:25.657738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:25.657738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 251/499] 10.222.0.61 () {56 vars in 1093 bytes} [Fri Aug 26 12:01:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:25.657738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15efe227-39ba-4ae1-9f27-51f5c718bcfb None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:25.659505 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b554de-8863-4826-9810-4c05e80760e2 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:25.659787 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b554de-8863-4826-9810-4c05e80760e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:25.660057 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b554de-8863-4826-9810-4c05e80760e2 None admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:25.660705 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b554de-8863-4826-9810-4c05e80760e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:25.661003 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b554de-8863-4826-9810-4c05e80760e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:25.662770 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b554de-8863-4826-9810-4c05e80760e2 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:25.680985 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 249/500] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:01:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:25.680985 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b554de-8863-4826-9810-4c05e80760e2 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:25.705224 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:25.782833 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15efe227-39ba-4ae1-9f27-51f5c718bcfb None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:25.790173 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b554de-8863-4826-9810-4c05e80760e2 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:25.810166 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:25.820234 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fe4f77-8b8e-438a-9b99-849223c53459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:25.820864 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fe4f77-8b8e-438a-9b99-849223c53459 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:25.821139 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fe4f77-8b8e-438a-9b99-849223c53459 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:25.821408 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fe4f77-8b8e-438a-9b99-849223c53459 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:25.827769 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a5a0da-abda-4f35-b612-bd698c1b7f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:25.828368 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a5a0da-abda-4f35-b612-bd698c1b7f1c None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:25.828645 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a5a0da-abda-4f35-b612-bd698c1b7f1c None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:25.828915 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a5a0da-abda-4f35-b612-bd698c1b7f1c None admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:25.829581 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a5a0da-abda-4f35-b612-bd698c1b7f1c None admin] RBAC: Adding query filter params (name=service) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:25.829880 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a5a0da-abda-4f35-b612-bd698c1b7f1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:25.831260 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a5a0da-abda-4f35-b612-bd698c1b7f1c None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:25.832844 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fe4f77-8b8e-438a-9b99-849223c53459 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:25.834284 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fe4f77-8b8e-438a-9b99-849223c53459 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:25.839288 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 250/501] 10.222.0.61 () {56 vars in 1103 bytes} [Fri Aug 26 12:01:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:25.840069 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a5a0da-abda-4f35-b612-bd698c1b7f1c None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:25.851663 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-91fe4f77-8b8e-438a-9b99-849223c53459 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:25.851663 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:25.851663 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:25.851663 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:25.851663 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:25.851663 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:25.851663 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:25.851663 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:25.851663 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:25.851663 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:25.851663 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:25.851663 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:25.851663 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:01:25.851663 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:01:25.851663 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:01:25.851663 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:01:25.852675 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:25.852675 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:25.852675 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:25.852675 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:25.852675 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:25.852675 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:25.852675 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:25.852675 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:25.852675 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:25.852675 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:25.852675 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:25.852675 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:25.852675 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:25.852675 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:25.852675 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:25.852675 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:25.852675 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:01:25.853558 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:01:25.853558 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:25.853558 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:25.853558 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 252/502] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:01:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:25.853558 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fe4f77-8b8e-438a-9b99-849223c53459 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:25.926085 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:25.946055 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a5a0da-abda-4f35-b612-bd698c1b7f1c None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:25.955787 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fe4f77-8b8e-438a-9b99-849223c53459 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:25.980033 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1020ad5e-c4f9-4dec-a7ae-82773a54d71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:25.980630 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020ad5e-c4f9-4dec-a7ae-82773a54d71a None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:25.980907 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020ad5e-c4f9-4dec-a7ae-82773a54d71a None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:25.981175 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020ad5e-c4f9-4dec-a7ae-82773a54d71a None admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:25.981950 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1020ad5e-c4f9-4dec-a7ae-82773a54d71a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:25.982250 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1020ad5e-c4f9-4dec-a7ae-82773a54d71a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:25.983820 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1020ad5e-c4f9-4dec-a7ae-82773a54d71a None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:25.989424 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2336d1f8-7c18-46d1-8051-4bafcde0a289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:25.990039 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2336d1f8-7c18-46d1-8051-4bafcde0a289 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:25.990320 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2336d1f8-7c18-46d1-8051-4bafcde0a289 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:25.990594 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2336d1f8-7c18-46d1-8051-4bafcde0a289 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:25.998160 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2336d1f8-7c18-46d1-8051-4bafcde0a289 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:25.999667 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2336d1f8-7c18-46d1-8051-4bafcde0a289 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:26.010857 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-2336d1f8-7c18-46d1-8051-4bafcde0a289 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 26 12:01:26.010857 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:26.010857 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:26.010857 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:26.010857 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:26.010857 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:26.010857 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:26.010857 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:26.010857 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:26.010857 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:26.010857 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:26.010857 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:26.010857 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 12:01:26.010857 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 12:01:26.010857 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 12:01:26.010857 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 12:01:26.011776 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:26.011776 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:26.011776 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 12:01:26.011776 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:01:26.011776 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 12:01:26.011776 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:01:26.011776 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:26.011776 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:26.011776 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:26.011776 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:26.011776 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:26.011776 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:26.011776 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:26.011776 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:26.011776 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:26.011776 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:26.011776 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:26.011776 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:26.012669 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:26.012669 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:26.012669 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 12:01:26.012669 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 12:01:26.012669 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 12:01:26.012669 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 12:01:26.012669 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 12:01:26.012669 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 12:01:26.012669 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 26 12:01:26.012669 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:26.012669 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 253/503] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:01:25 2022] GET /identity/v3/users/cinder => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:26.013752 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6cb572-f9f4-4305-ac74-d603f0d0373b None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:26.014272 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6cb572-f9f4-4305-ac74-d603f0d0373b None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:26.014533 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6cb572-f9f4-4305-ac74-d603f0d0373b None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:26.018280 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 254/504] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:01:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:01:26.018280 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6cb572-f9f4-4305-ac74-d603f0d0373b None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:26.021943 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 251/505] 10.222.0.61 () {56 vars in 1133 bytes} [Fri Aug 26 12:01:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:26.022752 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1020ad5e-c4f9-4dec-a7ae-82773a54d71a None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:26.058315 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:26.119758 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6cb572-f9f4-4305-ac74-d603f0d0373b None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:26.120918 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1020ad5e-c4f9-4dec-a7ae-82773a54d71a None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:26.153321 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6715e208-95e4-41b2-a902-80ebe34ac435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:26.154551 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6715e208-95e4-41b2-a902-80ebe34ac435 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:26.154910 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6715e208-95e4-41b2-a902-80ebe34ac435 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:26.155183 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6715e208-95e4-41b2-a902-80ebe34ac435 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:26.156183 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e65e815-79fe-4177-9131-8fdaa6f0b518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:26.156779 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e65e815-79fe-4177-9131-8fdaa6f0b518 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:26.157056 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e65e815-79fe-4177-9131-8fdaa6f0b518 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:26.157322 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e65e815-79fe-4177-9131-8fdaa6f0b518 None admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:26.157962 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e65e815-79fe-4177-9131-8fdaa6f0b518 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:26.158259 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e65e815-79fe-4177-9131-8fdaa6f0b518 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:26.159666 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e65e815-79fe-4177-9131-8fdaa6f0b518 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:26.165501 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6715e208-95e4-41b2-a902-80ebe34ac435 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:26.166926 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6715e208-95e4-41b2-a902-80ebe34ac435 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:26.170887 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 252/506] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:01:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:26.171529 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e65e815-79fe-4177-9131-8fdaa6f0b518 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:26.177398 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:26.181139 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-6715e208-95e4-41b2-a902-80ebe34ac435 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:26.181139 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:26.181139 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:26.181139 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:26.181139 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:26.181139 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:26.181139 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:26.181139 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:26.181139 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:26.181139 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:26.181139 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:26.181139 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:26.181139 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:01:26.181139 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:01:26.181139 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:01:26.181139 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:01:26.182051 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:26.182051 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:26.182051 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:26.182051 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:26.182051 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:26.182051 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:26.182051 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:26.182051 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:26.182051 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:26.182051 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:26.182051 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:26.182051 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:26.182051 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:26.182051 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:26.182051 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:26.182051 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:26.182051 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:01:26.182889 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:01:26.182889 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:26.182889 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:26.182889 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 255/507] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:01:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:26.183549 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a391ef86-bd31-488f-8728-7ada84c91df2 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:26.183657 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a391ef86-bd31-488f-8728-7ada84c91df2 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:26.183921 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a391ef86-bd31-488f-8728-7ada84c91df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:26.201878 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-a391ef86-bd31-488f-8728-7ada84c91df2 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:01:26.203056 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-a391ef86-bd31-488f-8728-7ada84c91df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:01:26.268750 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 256/508] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:01:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:01:26.269589 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a391ef86-bd31-488f-8728-7ada84c91df2 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:26.286600 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e65e815-79fe-4177-9131-8fdaa6f0b518 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:26.290379 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:26.320269 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e81503-01f9-446f-a7e3-554c9650d534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:26.320868 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e81503-01f9-446f-a7e3-554c9650d534 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:26.321149 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e81503-01f9-446f-a7e3-554c9650d534 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:26.321424 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e81503-01f9-446f-a7e3-554c9650d534 None admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:26.322142 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e81503-01f9-446f-a7e3-554c9650d534 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:26.322446 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e81503-01f9-446f-a7e3-554c9650d534 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:26.323909 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e81503-01f9-446f-a7e3-554c9650d534 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:26.346422 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 253/509] 10.222.0.61 () {56 vars in 1137 bytes} [Fri Aug 26 12:01:26 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:26.346422 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e81503-01f9-446f-a7e3-554c9650d534 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:26.378930 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a391ef86-bd31-488f-8728-7ada84c91df2 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:26.401627 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:26.416052 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e543c7-57c8-425f-87b5-137ec4b22915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:26.416655 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e543c7-57c8-425f-87b5-137ec4b22915 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:26.416931 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e543c7-57c8-425f-87b5-137ec4b22915 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:26.417198 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e543c7-57c8-425f-87b5-137ec4b22915 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:26.442341 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e81503-01f9-446f-a7e3-554c9650d534 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:26.477969 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd95583-5e10-49dc-bdfb-750cc0b983e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:26.478595 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd95583-5e10-49dc-bdfb-750cc0b983e9 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:26.478884 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd95583-5e10-49dc-bdfb-750cc0b983e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:26.479149 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd95583-5e10-49dc-bdfb-750cc0b983e9 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:26.486270 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd95583-5e10-49dc-bdfb-750cc0b983e9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:26.487747 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd95583-5e10-49dc-bdfb-750cc0b983e9 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:26.491998 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e543c7-57c8-425f-87b5-137ec4b22915 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:26.493688 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e543c7-57c8-425f-87b5-137ec4b22915 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:26.501027 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-7fd95583-5e10-49dc-bdfb-750cc0b983e9 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 26 12:01:26.501027 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:26.501027 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:26.501027 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:26.501027 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:26.501027 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:26.501027 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:26.501027 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:26.501027 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:26.501027 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:26.501027 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:26.501027 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:26.501027 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 12:01:26.501027 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 12:01:26.501027 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 12:01:26.501027 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 12:01:26.501968 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:26.501968 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:26.501968 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 12:01:26.501968 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:01:26.501968 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 12:01:26.501968 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:01:26.501968 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:26.501968 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:26.501968 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:26.501968 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:26.501968 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:26.501968 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:26.501968 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:26.501968 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:26.501968 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:26.501968 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:26.501968 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:26.501968 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:26.502898 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:26.502898 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:26.502898 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 12:01:26.502898 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 12:01:26.502898 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 12:01:26.502898 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 12:01:26.502898 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 12:01:26.502898 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 12:01:26.502898 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 26 12:01:26.502898 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:26.502898 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 254/510] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:01:26 2022] GET /identity/v3/users/glance => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:26.502898 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd95583-5e10-49dc-bdfb-750cc0b983e9 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:26.521870 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:26.604422 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 257/511] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:01:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 12:01:26.605071 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e543c7-57c8-425f-87b5-137ec4b22915 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:26.608209 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd95583-5e10-49dc-bdfb-750cc0b983e9 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:26.627924 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:26.643700 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950e7cf6-5308-4f9c-93e3-4ad9bc315e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:26.644243 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e7cf6-5308-4f9c-93e3-4ad9bc315e22 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:26.644524 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e7cf6-5308-4f9c-93e3-4ad9bc315e22 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:26.644799 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e7cf6-5308-4f9c-93e3-4ad9bc315e22 None admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:26.645439 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950e7cf6-5308-4f9c-93e3-4ad9bc315e22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:26.645745 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950e7cf6-5308-4f9c-93e3-4ad9bc315e22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:26.647141 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950e7cf6-5308-4f9c-93e3-4ad9bc315e22 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:26.659196 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 255/512] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:01:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:26.660007 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950e7cf6-5308-4f9c-93e3-4ad9bc315e22 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:26.707873 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e543c7-57c8-425f-87b5-137ec4b22915 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:26.727428 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:26.743913 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25faf6e-ed77-41bc-ba04-84df049d7c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:26.744510 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25faf6e-ed77-41bc-ba04-84df049d7c30 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:26.744789 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25faf6e-ed77-41bc-ba04-84df049d7c30 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:26.745065 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25faf6e-ed77-41bc-ba04-84df049d7c30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:26.765981 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950e7cf6-5308-4f9c-93e3-4ad9bc315e22 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:26.799268 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47619a6d-9e93-4770-87e1-665639347adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:26.799866 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47619a6d-9e93-4770-87e1-665639347adc None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:26.800144 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47619a6d-9e93-4770-87e1-665639347adc None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:26.800416 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47619a6d-9e93-4770-87e1-665639347adc None admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:26.801135 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47619a6d-9e93-4770-87e1-665639347adc None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:26.801439 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47619a6d-9e93-4770-87e1-665639347adc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:26.802861 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47619a6d-9e93-4770-87e1-665639347adc None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:26.812190 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25faf6e-ed77-41bc-ba04-84df049d7c30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:26.813645 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25faf6e-ed77-41bc-ba04-84df049d7c30 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:26.820169 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 256/513] 10.222.0.61 () {56 vars in 1137 bytes} [Fri Aug 26 12:01:26 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:26.820975 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47619a6d-9e93-4770-87e1-665639347adc None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:26.861839 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:26.909662 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 258/514] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:01:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 12:01:26.912088 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25faf6e-ed77-41bc-ba04-84df049d7c30 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:26.922353 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47619a6d-9e93-4770-87e1-665639347adc None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:26.932386 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:26.956538 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e5fd34-265d-46c3-ad0a-64fc756e3dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:26.957150 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e5fd34-265d-46c3-ad0a-64fc756e3dbd None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:26.957428 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e5fd34-265d-46c3-ad0a-64fc756e3dbd None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:26.957692 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e5fd34-265d-46c3-ad0a-64fc756e3dbd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:26.967670 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e5fd34-265d-46c3-ad0a-64fc756e3dbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:26.969290 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e5fd34-265d-46c3-ad0a-64fc756e3dbd None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:26.983983 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-c1e5fd34-265d-46c3-ad0a-64fc756e3dbd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:26.983983 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:26.983983 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:26.983983 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:26.983983 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:26.983983 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:26.983983 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:26.983983 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:26.983983 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:26.983983 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:26.983983 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:26.983983 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:26.983983 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:01:26.983983 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:01:26.983983 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:01:26.983983 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:01:26.984886 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:26.984886 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:26.984886 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:26.984886 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:26.984886 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:26.984886 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:26.984886 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:26.984886 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:26.984886 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:26.984886 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:26.984886 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:26.984886 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:26.984886 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:26.984886 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:26.984886 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:26.984886 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:26.984886 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:01:26.985731 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:01:26.985731 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:26.985731 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:26.985731 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 257/515] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:01:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:26.985954 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e5fd34-265d-46c3-ad0a-64fc756e3dbd None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:27.011938 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25faf6e-ed77-41bc-ba04-84df049d7c30 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:27.045498 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777e8d85-cf11-49f5-b3c7-213ef293e4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:27.046113 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e8d85-cf11-49f5-b3c7-213ef293e4e9 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:27.046390 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e8d85-cf11-49f5-b3c7-213ef293e4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:27.046659 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e8d85-cf11-49f5-b3c7-213ef293e4e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:27.056286 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777e8d85-cf11-49f5-b3c7-213ef293e4e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:27.057771 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777e8d85-cf11-49f5-b3c7-213ef293e4e9 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:27.072655 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-777e8d85-cf11-49f5-b3c7-213ef293e4e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:27.072655 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:27.072655 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:27.072655 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:27.072655 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:27.072655 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:27.072655 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:27.072655 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:27.072655 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:27.072655 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:27.072655 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:27.072655 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:27.072655 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:01:27.072655 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:01:27.072655 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:01:27.072655 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:01:27.073699 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:27.073699 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:27.073699 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:27.073699 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:27.073699 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:27.073699 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:27.073699 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:27.073699 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:27.073699 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:27.073699 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:27.073699 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:27.073699 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:27.073699 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:27.073699 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:27.073699 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:27.073699 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:27.073699 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:01:27.074566 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:01:27.074566 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:27.074566 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:27.074566 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 259/516] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:01:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:27.074566 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777e8d85-cf11-49f5-b3c7-213ef293e4e9 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:27.078371 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:27.089024 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e5fd34-265d-46c3-ad0a-64fc756e3dbd None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:27.121906 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edd1330-30d0-4e3c-98ac-17ee09a97e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:27.123062 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd1330-30d0-4e3c-98ac-17ee09a97e38 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:27.123370 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd1330-30d0-4e3c-98ac-17ee09a97e38 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:27.123632 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd1330-30d0-4e3c-98ac-17ee09a97e38 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:27.131554 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edd1330-30d0-4e3c-98ac-17ee09a97e38 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:27.133767 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edd1330-30d0-4e3c-98ac-17ee09a97e38 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:27.145262 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-6edd1330-30d0-4e3c-98ac-17ee09a97e38 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 26 12:01:27.145262 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:27.145262 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:27.145262 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:27.145262 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:27.145262 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:27.145262 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:27.145262 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:27.145262 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:27.145262 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:27.145262 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:27.145262 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:27.145262 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 12:01:27.145262 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 12:01:27.145262 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 12:01:27.145262 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 12:01:27.146346 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:27.146346 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:27.146346 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 12:01:27.146346 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:01:27.146346 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 12:01:27.146346 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:01:27.146346 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:27.146346 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:27.146346 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:27.146346 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:27.146346 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:27.146346 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:27.146346 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:27.146346 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:27.146346 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:27.146346 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:27.146346 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:27.146346 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:27.147259 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:27.147259 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:27.147259 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 12:01:27.147259 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 12:01:27.147259 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 12:01:27.147259 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 12:01:27.147259 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 12:01:27.147259 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 12:01:27.147259 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 26 12:01:27.147259 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:27.147259 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 258/517] 10.222.0.61 () {56 vars in 1102 bytes} [Fri Aug 26 12:01:26 2022] GET /identity/v3/users/neutron => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:27.147826 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edd1330-30d0-4e3c-98ac-17ee09a97e38 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:27.152316 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:27.183472 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777e8d85-cf11-49f5-b3c7-213ef293e4e9 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:27.217537 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fad1461-762b-407d-8f7b-69cc0dee7d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:27.218129 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad1461-762b-407d-8f7b-69cc0dee7d08 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:27.218408 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad1461-762b-407d-8f7b-69cc0dee7d08 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:27.218680 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad1461-762b-407d-8f7b-69cc0dee7d08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:27.244602 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edd1330-30d0-4e3c-98ac-17ee09a97e38 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:27.278772 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107aaddb-9358-4f7d-91cc-67d8e962c303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:27.279422 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107aaddb-9358-4f7d-91cc-67d8e962c303 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:27.279707 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107aaddb-9358-4f7d-91cc-67d8e962c303 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:27.279980 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107aaddb-9358-4f7d-91cc-67d8e962c303 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:27.285412 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fad1461-762b-407d-8f7b-69cc0dee7d08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:27.286877 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fad1461-762b-407d-8f7b-69cc0dee7d08 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:27.290316 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107aaddb-9358-4f7d-91cc-67d8e962c303 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:27.291775 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107aaddb-9358-4f7d-91cc-67d8e962c303 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:27.305443 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-107aaddb-9358-4f7d-91cc-67d8e962c303 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:27.305443 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:27.305443 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:27.305443 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:27.305443 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:27.305443 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:27.305443 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:27.305443 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:27.305443 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:27.305443 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:27.305443 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:27.305443 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:27.305443 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:01:27.305443 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:01:27.305443 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:01:27.305443 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:01:27.306347 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:27.306347 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:27.306347 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:27.306347 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:27.306347 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:27.306347 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:27.306347 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:27.306347 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:27.306347 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:27.306347 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:27.306347 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:27.306347 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:27.306347 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:27.306347 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:27.306347 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:27.306347 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:27.306347 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:01:27.307229 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:01:27.307229 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:27.307229 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:27.307229 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 259/518] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:01:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:27.307464 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107aaddb-9358-4f7d-91cc-67d8e962c303 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:27.314247 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:27.355681 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:27.386863 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 260/519] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:01:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 12:01:27.387558 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fad1461-762b-407d-8f7b-69cc0dee7d08 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:27.416406 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107aaddb-9358-4f7d-91cc-67d8e962c303 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:27.452292 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01985ed-bef3-4630-b12f-d8bcae1371bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:27.452819 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01985ed-bef3-4630-b12f-d8bcae1371bf None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:27.453099 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01985ed-bef3-4630-b12f-d8bcae1371bf None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:27.453377 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01985ed-bef3-4630-b12f-d8bcae1371bf None admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:27.454013 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01985ed-bef3-4630-b12f-d8bcae1371bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:27.454343 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01985ed-bef3-4630-b12f-d8bcae1371bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:27.455748 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01985ed-bef3-4630-b12f-d8bcae1371bf None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:27.467361 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 260/520] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:01:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:27.468184 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01985ed-bef3-4630-b12f-d8bcae1371bf None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:27.493485 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fad1461-762b-407d-8f7b-69cc0dee7d08 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:27.528325 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e9434-1e1d-4c3d-a59d-e931cc8e4612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:27.529081 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e9434-1e1d-4c3d-a59d-e931cc8e4612 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:27.529358 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e9434-1e1d-4c3d-a59d-e931cc8e4612 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:27.529627 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e9434-1e1d-4c3d-a59d-e931cc8e4612 None admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:27.530253 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9e9434-1e1d-4c3d-a59d-e931cc8e4612 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:27.530554 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9e9434-1e1d-4c3d-a59d-e931cc8e4612 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:27.531931 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9e9434-1e1d-4c3d-a59d-e931cc8e4612 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:27.532028 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:27.543659 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 261/521] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:01:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:27.545006 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e9434-1e1d-4c3d-a59d-e931cc8e4612 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:27.589544 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:27.593083 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01985ed-bef3-4630-b12f-d8bcae1371bf None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:27.627116 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d681655-c8a5-4e51-9d1f-7220eeb8a233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:27.627735 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d681655-c8a5-4e51-9d1f-7220eeb8a233 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:27.628010 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d681655-c8a5-4e51-9d1f-7220eeb8a233 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:27.628278 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d681655-c8a5-4e51-9d1f-7220eeb8a233 None admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:27.628989 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d681655-c8a5-4e51-9d1f-7220eeb8a233 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:27.629289 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d681655-c8a5-4e51-9d1f-7220eeb8a233 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:27.630700 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d681655-c8a5-4e51-9d1f-7220eeb8a233 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:27.648685 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 261/522] 10.222.0.61 () {56 vars in 1139 bytes} [Fri Aug 26 12:01:27 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:27.649479 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d681655-c8a5-4e51-9d1f-7220eeb8a233 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:27.660650 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e9434-1e1d-4c3d-a59d-e931cc8e4612 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:27.694354 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816c70ee-e3b7-4e03-a0c2-f92f6bccc417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:27.695108 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816c70ee-e3b7-4e03-a0c2-f92f6bccc417 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:27.695767 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816c70ee-e3b7-4e03-a0c2-f92f6bccc417 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:27.696052 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816c70ee-e3b7-4e03-a0c2-f92f6bccc417 None admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:27.696693 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816c70ee-e3b7-4e03-a0c2-f92f6bccc417 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:27.696996 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816c70ee-e3b7-4e03-a0c2-f92f6bccc417 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:27.698361 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816c70ee-e3b7-4e03-a0c2-f92f6bccc417 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:27.714064 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 262/523] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:01:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:27.714064 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816c70ee-e3b7-4e03-a0c2-f92f6bccc417 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:27.751644 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d681655-c8a5-4e51-9d1f-7220eeb8a233 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:27.771580 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:27.786954 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef646f45-0c29-46ae-88e6-86c8ec0f09f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:27.787604 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef646f45-0c29-46ae-88e6-86c8ec0f09f2 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:27.787880 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef646f45-0c29-46ae-88e6-86c8ec0f09f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:27.788148 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef646f45-0c29-46ae-88e6-86c8ec0f09f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:27.798538 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef646f45-0c29-46ae-88e6-86c8ec0f09f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:27.799995 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef646f45-0c29-46ae-88e6-86c8ec0f09f2 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:27.804837 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:27.815403 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-ef646f45-0c29-46ae-88e6-86c8ec0f09f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:27.815403 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:27.815403 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:27.815403 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:27.815403 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:27.815403 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:27.815403 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:27.815403 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:27.815403 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:27.815403 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:27.815403 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:27.815403 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:27.815403 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:01:27.815403 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:01:27.815403 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:01:27.815403 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:01:27.816587 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:27.816587 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:27.816587 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:27.816587 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:27.816587 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:27.816587 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:27.816587 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:27.816587 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:27.816587 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:27.816587 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:27.816587 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:27.816587 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:27.816587 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:27.816587 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:27.816587 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:27.816587 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:27.816587 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:01:27.817456 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:01:27.817456 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:27.817456 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:27.817627 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816c70ee-e3b7-4e03-a0c2-f92f6bccc417 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:27.817697 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 262/524] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:01:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:27.817697 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef646f45-0c29-46ae-88e6-86c8ec0f09f2 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:27.851656 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39277433-7b58-4bca-9b29-9f368c7010b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:27.852249 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39277433-7b58-4bca-9b29-9f368c7010b6 None admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:27.852535 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39277433-7b58-4bca-9b29-9f368c7010b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:27.852806 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39277433-7b58-4bca-9b29-9f368c7010b6 None admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:27.853439 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39277433-7b58-4bca-9b29-9f368c7010b6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:27.854860 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39277433-7b58-4bca-9b29-9f368c7010b6 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:27.875845 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 263/525] 10.222.0.61 () {60 vars in 1132 bytes} [Fri Aug 26 12:01:27 2022] POST /identity/v3/users => generated 247 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:01:27.876714 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39277433-7b58-4bca-9b29-9f368c7010b6 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:27.935538 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef646f45-0c29-46ae-88e6-86c8ec0f09f2 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:27.976528 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58e16f4-1133-4b8c-a0c2-15af875bc557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:27.977150 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e16f4-1133-4b8c-a0c2-15af875bc557 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:27.977360 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e16f4-1133-4b8c-a0c2-15af875bc557 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:27.977617 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e16f4-1133-4b8c-a0c2-15af875bc557 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:27.985945 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58e16f4-1133-4b8c-a0c2-15af875bc557 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:27.987494 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58e16f4-1133-4b8c-a0c2-15af875bc557 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:27.992515 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39277433-7b58-4bca-9b29-9f368c7010b6 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:28.000880 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-d58e16f4-1133-4b8c-a0c2-15af875bc557 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:01:28.000880 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:28.000880 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:28.000880 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:28.000880 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:28.000880 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:28.000880 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:28.000880 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:28.000880 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:28.000880 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:28.000880 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:28.000880 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:28.000880 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 12:01:28.000880 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 12:01:28.000880 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 12:01:28.002000 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 12:01:28.002000 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:28.002000 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:28.002000 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:28.002000 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:28.002000 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:28.002000 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:28.002000 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:28.002000 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:28.002000 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:28.002000 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:28.002000 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:28.002000 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:28.002000 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:28.002000 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:28.002000 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:28.002000 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:28.003062 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 12:01:28.003062 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 12:01:28.003062 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 12:01:28.003062 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 12:01:28.003062 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 12:01:28.003062 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 12:01:28.003062 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:01:28.003062 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:28.007072 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 263/526] 10.222.0.61 () {56 vars in 1111 bytes} [Fri Aug 26 12:01:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:28.008274 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58e16f4-1133-4b8c-a0c2-15af875bc557 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:28.035448 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e182ad-09c2-4dea-b97b-c9aa1dd202ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:28.036222 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e182ad-09c2-4dea-b97b-c9aa1dd202ef None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:28.036564 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e182ad-09c2-4dea-b97b-c9aa1dd202ef None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:28.036841 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e182ad-09c2-4dea-b97b-c9aa1dd202ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:28.042613 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:28.058720 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:28.119160 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e182ad-09c2-4dea-b97b-c9aa1dd202ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:28.120258 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58e16f4-1133-4b8c-a0c2-15af875bc557 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:28.120667 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e182ad-09c2-4dea-b97b-c9aa1dd202ef None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:28.156743 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f03dd0a-867d-4bda-af19-7e17a49ee8e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:28.157488 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f03dd0a-867d-4bda-af19-7e17a49ee8e4 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:28.157765 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f03dd0a-867d-4bda-af19-7e17a49ee8e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:28.158035 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f03dd0a-867d-4bda-af19-7e17a49ee8e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:28.165022 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f03dd0a-867d-4bda-af19-7e17a49ee8e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:28.166603 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f03dd0a-867d-4bda-af19-7e17a49ee8e4 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:28.179107 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-3f03dd0a-867d-4bda-af19-7e17a49ee8e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:01:28.179107 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:28.179107 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:28.179107 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:28.179107 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:28.179107 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:28.179107 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:28.179107 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:28.179107 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:28.179107 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:28.179107 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:28.179107 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:28.179107 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 12:01:28.179107 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 12:01:28.179107 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 12:01:28.179985 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 12:01:28.179985 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:28.179985 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:28.179985 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:28.179985 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:28.179985 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:28.179985 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:28.179985 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:28.179985 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:28.179985 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:28.179985 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:28.179985 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:28.179985 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:28.179985 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:28.179985 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:28.179985 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:28.179985 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:28.180908 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 12:01:28.180908 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 12:01:28.180908 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 12:01:28.180908 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 12:01:28.180908 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 12:01:28.180908 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 12:01:28.180908 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:01:28.180908 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:28.180908 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 264/527] 10.222.0.61 () {56 vars in 1111 bytes} [Fri Aug 26 12:01:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:28.181366 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f03dd0a-867d-4bda-af19-7e17a49ee8e4 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:28.228730 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 264/528] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:01:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 12:01:28.229316 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e182ad-09c2-4dea-b97b-c9aa1dd202ef None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:28.277045 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:28.287849 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:28.298868 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f03dd0a-867d-4bda-af19-7e17a49ee8e4 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:28.335687 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0af525-5320-4248-8c0e-5058bc2f50fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:28.336291 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0af525-5320-4248-8c0e-5058bc2f50fb None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:28.336419 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e182ad-09c2-4dea-b97b-c9aa1dd202ef None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:28.336598 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0af525-5320-4248-8c0e-5058bc2f50fb None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:28.336838 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0af525-5320-4248-8c0e-5058bc2f50fb None admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:28.337476 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0af525-5320-4248-8c0e-5058bc2f50fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:28.338033 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0af525-5320-4248-8c0e-5058bc2f50fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:28.340344 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0af525-5320-4248-8c0e-5058bc2f50fb None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:28.360920 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 265/529] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:01:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:28.361728 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0af525-5320-4248-8c0e-5058bc2f50fb None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:28.374552 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c318fbe-c3f2-48fa-9458-f45ae4ee878d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:28.375163 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c318fbe-c3f2-48fa-9458-f45ae4ee878d None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:28.375518 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c318fbe-c3f2-48fa-9458-f45ae4ee878d None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:28.375740 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c318fbe-c3f2-48fa-9458-f45ae4ee878d None admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:28.376423 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c318fbe-c3f2-48fa-9458-f45ae4ee878d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:28.376729 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c318fbe-c3f2-48fa-9458-f45ae4ee878d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:28.378163 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c318fbe-c3f2-48fa-9458-f45ae4ee878d None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:28.402171 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 265/530] 10.222.0.61 () {56 vars in 1148 bytes} [Fri Aug 26 12:01:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:28.402171 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c318fbe-c3f2-48fa-9458-f45ae4ee878d None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:28.480275 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0af525-5320-4248-8c0e-5058bc2f50fb None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:28.500138 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c318fbe-c3f2-48fa-9458-f45ae4ee878d None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:28.509663 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:28.518486 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a87b8-3f9c-4b2b-aaaf-3844352f710d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:28.519316 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a87b8-3f9c-4b2b-aaaf-3844352f710d None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:28.519674 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a87b8-3f9c-4b2b-aaaf-3844352f710d None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:28.519952 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a87b8-3f9c-4b2b-aaaf-3844352f710d None admin] PATH_INFO: `/v3/projects` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:28.520650 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615a87b8-3f9c-4b2b-aaaf-3844352f710d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:28.520957 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615a87b8-3f9c-4b2b-aaaf-3844352f710d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:28.522504 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615a87b8-3f9c-4b2b-aaaf-3844352f710d None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:28.535096 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4989a4ad-c789-4e70-951f-d96b1258efe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:28.535752 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4989a4ad-c789-4e70-951f-d96b1258efe0 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:28.536023 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4989a4ad-c789-4e70-951f-d96b1258efe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:28.536289 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4989a4ad-c789-4e70-951f-d96b1258efe0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:28.546008 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 266/531] 10.222.0.61 () {56 vars in 1148 bytes} [Fri Aug 26 12:01:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:28.546008 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a87b8-3f9c-4b2b-aaaf-3844352f710d None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:28.551454 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4989a4ad-c789-4e70-951f-d96b1258efe0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:28.552986 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4989a4ad-c789-4e70-951f-d96b1258efe0 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:28.561875 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:28.568952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-4989a4ad-c789-4e70-951f-d96b1258efe0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:28.568952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:28.568952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:28.568952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:28.568952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:28.568952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:28.568952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:28.568952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:28.568952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:28.568952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:28.568952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:28.568952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:28.568952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:01:28.568952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:01:28.568952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:01:28.568952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:01:28.569849 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:28.569849 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:28.569849 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:28.569849 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:28.569849 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:28.569849 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:28.569849 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:28.569849 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:28.569849 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:28.569849 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:28.569849 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:28.569849 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:28.569849 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:28.569849 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:28.569849 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:28.569849 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:28.569849 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:01:28.570704 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:01:28.570704 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:28.570704 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:28.574608 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 266/532] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:01:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:28.576084 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4989a4ad-c789-4e70-951f-d96b1258efe0 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:28.653253 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a87b8-3f9c-4b2b-aaaf-3844352f710d None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:28.688778 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4989a4ad-c789-4e70-951f-d96b1258efe0 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:28.691623 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369a53c5-f51b-4788-a6d6-9bc824b48cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:28.691753 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369a53c5-f51b-4788-a6d6-9bc824b48cfa None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:28.692006 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369a53c5-f51b-4788-a6d6-9bc824b48cfa None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:28.692272 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369a53c5-f51b-4788-a6d6-9bc824b48cfa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:28.698884 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369a53c5-f51b-4788-a6d6-9bc824b48cfa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:28.700422 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369a53c5-f51b-4788-a6d6-9bc824b48cfa None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:28.711439 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-369a53c5-f51b-4788-a6d6-9bc824b48cfa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:01:28.711439 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:28.711439 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:28.711439 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:28.711439 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:28.711439 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:28.711439 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:28.711439 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:28.711439 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:28.711439 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:28.711439 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:28.711439 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:28.711439 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 12:01:28.711439 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 12:01:28.711439 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 12:01:28.712325 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 12:01:28.712325 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:28.712325 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:28.712325 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:28.712325 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:28.712325 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:28.712325 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:28.712325 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:28.712325 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:28.712325 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:28.712325 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:28.712325 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:28.712325 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:28.712325 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:28.712325 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:28.712325 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:28.712325 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:28.713185 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 12:01:28.713185 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 12:01:28.713185 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 12:01:28.713185 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 12:01:28.713185 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 12:01:28.713185 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 12:01:28.713185 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:01:28.713185 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:28.713185 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 267/533] 10.222.0.61 () {56 vars in 1111 bytes} [Fri Aug 26 12:01:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:28.713642 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369a53c5-f51b-4788-a6d6-9bc824b48cfa None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:28.730007 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758968d0-ee92-4bc1-b1b0-2ae122ab85fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:28.730421 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758968d0-ee92-4bc1-b1b0-2ae122ab85fa None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:28.730647 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758968d0-ee92-4bc1-b1b0-2ae122ab85fa None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:28.730919 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758968d0-ee92-4bc1-b1b0-2ae122ab85fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:28.731716 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758968d0-ee92-4bc1-b1b0-2ae122ab85fa None admin] RBAC: Adding query filter params (role.id=519347d2bd2c4dc1aadb0b9f8c3c8e3d, scope.project.id=ecd68423a83a427681d44dbf266fce6e, user.id=7aacd7d6d17e4f10ba5c9107a12beafa) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:28.732528 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758968d0-ee92-4bc1-b1b0-2ae122ab85fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:28.734107 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758968d0-ee92-4bc1-b1b0-2ae122ab85fa None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:28.752242 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 267/534] 10.222.0.61 () {56 vars in 1374 bytes} [Fri Aug 26 12:01:28 2022] GET /identity/v3/role_assignments?user.id=7aacd7d6d17e4f10ba5c9107a12beafa&scope.project.id=ecd68423a83a427681d44dbf266fce6e&role.id=519347d2bd2c4dc1aadb0b9f8c3c8e3d => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:28.753450 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758968d0-ee92-4bc1-b1b0-2ae122ab85fa None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:28.793729 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:28.803702 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:28.852817 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369a53c5-f51b-4788-a6d6-9bc824b48cfa None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:28.884364 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758968d0-ee92-4bc1-b1b0-2ae122ab85fa None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:28.901157 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a0812-0bbc-410a-b45c-13fa9964687f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:28.902116 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a0812-0bbc-410a-b45c-13fa9964687f None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:28.902745 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a0812-0bbc-410a-b45c-13fa9964687f None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:28.903340 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a0812-0bbc-410a-b45c-13fa9964687f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:28.904859 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777a0812-0bbc-410a-b45c-13fa9964687f None admin] RBAC: Adding query filter params (role.id=519347d2bd2c4dc1aadb0b9f8c3c8e3d, scope.project.id=ecd68423a83a427681d44dbf266fce6e, user.id=51239ac314e245e8a5706e65b133db3b) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:28.905390 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777a0812-0bbc-410a-b45c-13fa9964687f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:28.907195 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777a0812-0bbc-410a-b45c-13fa9964687f None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:28.914086 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 268/535] 10.222.0.61 () {56 vars in 1374 bytes} [Fri Aug 26 12:01:28 2022] GET /identity/v3/role_assignments?user.id=51239ac314e245e8a5706e65b133db3b&scope.project.id=ecd68423a83a427681d44dbf266fce6e&role.id=519347d2bd2c4dc1aadb0b9f8c3c8e3d => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:28.915277 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a0812-0bbc-410a-b45c-13fa9964687f None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:28.922700 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24deadac-ff44-453c-ad5a-171373c10282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:28.923371 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24deadac-ff44-453c-ad5a-171373c10282 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:28.923659 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24deadac-ff44-453c-ad5a-171373c10282 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:28.923936 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24deadac-ff44-453c-ad5a-171373c10282 None admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:28.924603 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24deadac-ff44-453c-ad5a-171373c10282 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:28.924910 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24deadac-ff44-453c-ad5a-171373c10282 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:28.926368 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24deadac-ff44-453c-ad5a-171373c10282 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:28.942725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 268/536] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:01:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:28.947921 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24deadac-ff44-453c-ad5a-171373c10282 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:29.038375 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a0812-0bbc-410a-b45c-13fa9964687f None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:29.081373 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:29.089518 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591ecc3a-0ce1-48a0-b05c-f13c6633dad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:29.090289 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ecc3a-0ce1-48a0-b05c-f13c6633dad5 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:29.090665 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ecc3a-0ce1-48a0-b05c-f13c6633dad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:29.091694 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ecc3a-0ce1-48a0-b05c-f13c6633dad5 None admin] PATH_INFO: `/v3/projects` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:29.092544 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591ecc3a-0ce1-48a0-b05c-f13c6633dad5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:29.092947 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591ecc3a-0ce1-48a0-b05c-f13c6633dad5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:29.094535 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591ecc3a-0ce1-48a0-b05c-f13c6633dad5 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:29.103408 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:29.111596 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24deadac-ff44-453c-ad5a-171373c10282 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:29.112103 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 269/537] 10.222.0.61 () {56 vars in 1148 bytes} [Fri Aug 26 12:01:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:29.121551 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591ecc3a-0ce1-48a0-b05c-f13c6633dad5 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:29.175986 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaf6d31-e189-44a9-80d0-102250198fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:29.177194 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf6d31-e189-44a9-80d0-102250198fba None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:29.177640 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf6d31-e189-44a9-80d0-102250198fba None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:29.178004 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf6d31-e189-44a9-80d0-102250198fba None admin] PATH_INFO: `/v3/projects/service` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:29.188752 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaf6d31-e189-44a9-80d0-102250198fba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:29.190379 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaf6d31-e189-44a9-80d0-102250198fba None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:29.207552 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-acaf6d31-e189-44a9-80d0-102250198fba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:01:29.207552 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:29.207552 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:29.207552 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:29.207552 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:29.207552 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:29.207552 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:29.207552 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:29.207552 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:29.207552 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:29.207552 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:29.207552 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:29.207552 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 12:01:29.207552 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 12:01:29.207552 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 12:01:29.208681 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 12:01:29.208681 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:29.208681 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:29.208681 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:29.208681 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:29.208681 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:29.208681 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:29.208681 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:29.208681 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:29.208681 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:29.208681 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:29.208681 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:29.208681 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:29.208681 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:29.208681 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:29.208681 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:29.208681 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:29.209751 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 12:01:29.209751 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 12:01:29.209751 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 12:01:29.209751 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 12:01:29.209751 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 12:01:29.209751 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 12:01:29.209751 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:01:29.209751 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:29.215223 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 269/538] 10.222.0.61 () {56 vars in 1111 bytes} [Fri Aug 26 12:01:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:29.218544 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaf6d31-e189-44a9-80d0-102250198fba None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:29.279962 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591ecc3a-0ce1-48a0-b05c-f13c6633dad5 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:29.334336 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaf6d31-e189-44a9-80d0-102250198fba None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:29.364467 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3e1ae8-6ed4-4a73-9fdd-5da8c9e6574a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:29.365349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3e1ae8-6ed4-4a73-9fdd-5da8c9e6574a None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:29.365762 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3e1ae8-6ed4-4a73-9fdd-5da8c9e6574a None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:29.366176 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3e1ae8-6ed4-4a73-9fdd-5da8c9e6574a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:29.367052 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3e1ae8-6ed4-4a73-9fdd-5da8c9e6574a None admin] RBAC: Adding query filter params (role.id=519347d2bd2c4dc1aadb0b9f8c3c8e3d, scope.project.id=ecd68423a83a427681d44dbf266fce6e, user.id=152fd7c708114f249b9c960165f0fdfe) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:29.367530 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3e1ae8-6ed4-4a73-9fdd-5da8c9e6574a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:29.369118 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3e1ae8-6ed4-4a73-9fdd-5da8c9e6574a None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:29.379225 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9558e71-cc98-49b3-bb10-37289546f849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:29.379225 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9558e71-cc98-49b3-bb10-37289546f849 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:29.379225 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9558e71-cc98-49b3-bb10-37289546f849 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:29.379225 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9558e71-cc98-49b3-bb10-37289546f849 None admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:29.379604 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9558e71-cc98-49b3-bb10-37289546f849 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:29.379604 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9558e71-cc98-49b3-bb10-37289546f849 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:29.379604 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9558e71-cc98-49b3-bb10-37289546f849 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:29.384547 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 270/539] 10.222.0.61 () {56 vars in 1374 bytes} [Fri Aug 26 12:01:29 2022] GET /identity/v3/role_assignments?user.id=152fd7c708114f249b9c960165f0fdfe&scope.project.id=ecd68423a83a427681d44dbf266fce6e&role.id=519347d2bd2c4dc1aadb0b9f8c3c8e3d => generated 259 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:29.391691 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 270/540] 10.222.0.61 () {56 vars in 1148 bytes} [Fri Aug 26 12:01:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:29.397834 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3e1ae8-6ed4-4a73-9fdd-5da8c9e6574a None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:29.463611 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:29.517953 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3e1ae8-6ed4-4a73-9fdd-5da8c9e6574a None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:29.556074 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ef6bf7-9208-41df-8c31-9100d25025a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:29.556684 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef6bf7-9208-41df-8c31-9100d25025a1 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:29.556961 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef6bf7-9208-41df-8c31-9100d25025a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:29.557233 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef6bf7-9208-41df-8c31-9100d25025a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:29.557961 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ef6bf7-9208-41df-8c31-9100d25025a1 None admin] RBAC: Adding query filter params (role.id=519347d2bd2c4dc1aadb0b9f8c3c8e3d, scope.project.id=ecd68423a83a427681d44dbf266fce6e, user.id=b0972dbd5cff4610b239326425e6351c) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:29.558293 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ef6bf7-9208-41df-8c31-9100d25025a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:29.560437 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ef6bf7-9208-41df-8c31-9100d25025a1 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:29.572827 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 271/541] 10.222.0.61 () {56 vars in 1374 bytes} [Fri Aug 26 12:01:29 2022] GET /identity/v3/role_assignments?user.id=b0972dbd5cff4610b239326425e6351c&scope.project.id=ecd68423a83a427681d44dbf266fce6e&role.id=519347d2bd2c4dc1aadb0b9f8c3c8e3d => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:29.995193 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8bd7cd-7f82-4562-858b-c74a5ad956f2 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:29.995593 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8bd7cd-7f82-4562-858b-c74a5ad956f2 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:29.995787 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8bd7cd-7f82-4562-858b-c74a5ad956f2 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:29.997557 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 271/542] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:01:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:01:30.003503 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e10c164-92a9-462f-a506-297560433d50 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:30.003725 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e10c164-92a9-462f-a506-297560433d50 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:30.004052 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e10c164-92a9-462f-a506-297560433d50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:30.022875 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-0e10c164-92a9-462f-a506-297560433d50 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:01:30.024151 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-0e10c164-92a9-462f-a506-297560433d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:01:30.108406 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 272/543] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:01:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:01:30.411585 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8bd7cd-7f82-4562-858b-c74a5ad956f2 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:30.415559 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:30.528033 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8bd7cd-7f82-4562-858b-c74a5ad956f2 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:30.564850 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e62a724-680b-4225-b1cb-a32cb753b927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:30.565475 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62a724-680b-4225-b1cb-a32cb753b927 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:30.565752 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62a724-680b-4225-b1cb-a32cb753b927 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:30.566072 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62a724-680b-4225-b1cb-a32cb753b927 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:30.581315 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e62a724-680b-4225-b1cb-a32cb753b927 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:01:30.586037 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-6e62a724-680b-4225-b1cb-a32cb753b927 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 12:01:30.586037 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:30.586037 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:30.586037 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:30.586037 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:30.586037 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:30.586037 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:30.586037 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:30.586037 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:30.586037 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:30.586037 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:30.586037 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:30.586037 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:01:30.586037 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:01:30.586037 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:01:30.586037 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:01:30.587952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:01:30.587952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:01:30.587952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:01:30.587952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:01:30.587952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:30.587952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:30.587952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:30.587952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:30.587952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:30.587952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:30.587952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:30.587952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:30.587952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:30.587952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:30.587952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:30.587952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:30.589093 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:30.589093 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:30.589093 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:30.589093 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:30.589093 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:01:30.589093 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:01:30.589093 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:01:30.589093 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:01:30.589093 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:01:30.589093 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:01:30.589093 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 12:01:30.589093 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:30.589093 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 272/544] 10.222.0.61 () {56 vars in 1102 bytes} [Fri Aug 26 12:01:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:30.593898 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e10c164-92a9-462f-a506-297560433d50 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:30.681949 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:30.758600 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e10c164-92a9-462f-a506-297560433d50 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:30.795081 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a1423c-80ac-4f5d-923f-1beb2be10389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:30.795991 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a1423c-80ac-4f5d-923f-1beb2be10389 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:30.796273 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a1423c-80ac-4f5d-923f-1beb2be10389 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:30.796555 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a1423c-80ac-4f5d-923f-1beb2be10389 None admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:30.797231 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a1423c-80ac-4f5d-923f-1beb2be10389 None admin] RBAC: Adding query filter params (name=service) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:30.797535 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a1423c-80ac-4f5d-923f-1beb2be10389 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:30.798908 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a1423c-80ac-4f5d-923f-1beb2be10389 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:30.807435 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 273/545] 10.222.0.61 () {56 vars in 1103 bytes} [Fri Aug 26 12:01:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:30.812683 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e62a724-680b-4225-b1cb-a32cb753b927 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:30.913366 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:30.955019 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e62a724-680b-4225-b1cb-a32cb753b927 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:30.975761 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc866c7-a382-4add-851b-a9b41a9c2c77 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:30.976352 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc866c7-a382-4add-851b-a9b41a9c2c77 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:30.976847 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc866c7-a382-4add-851b-a9b41a9c2c77 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:30.979585 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 274/546] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:01:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 12:01:30.983690 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c040cf44-a3e6-4e24-b8d6-b1769788b140 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:30.984205 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c040cf44-a3e6-4e24-b8d6-b1769788b140 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:30.984727 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c040cf44-a3e6-4e24-b8d6-b1769788b140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:30.997163 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e522db3-6917-4477-abbf-be1f94156d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:30.999562 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e522db3-6917-4477-abbf-be1f94156d99 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:31.000831 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e522db3-6917-4477-abbf-be1f94156d99 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:31.001388 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e522db3-6917-4477-abbf-be1f94156d99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:31.013608 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-c040cf44-a3e6-4e24-b8d6-b1769788b140 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:01:31.015176 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e522db3-6917-4477-abbf-be1f94156d99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:31.015615 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-c040cf44-a3e6-4e24-b8d6-b1769788b140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:01:31.016657 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e522db3-6917-4477-abbf-be1f94156d99 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:31.046953 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-0e522db3-6917-4477-abbf-be1f94156d99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:31.046953 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:31.046953 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:31.046953 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:31.046953 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:31.046953 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:31.046953 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:31.046953 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:31.046953 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:31.046953 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:31.046953 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:31.046953 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:31.046953 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:01:31.046953 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:01:31.046953 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:01:31.046953 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:01:31.047928 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:31.047928 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:31.047928 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:31.047928 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:31.047928 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:31.047928 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:31.047928 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:31.047928 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:31.047928 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:31.047928 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:31.047928 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:31.047928 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:31.047928 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:31.047928 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:31.047928 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:31.047928 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:31.047928 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:01:31.048770 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:01:31.048770 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:31.048770 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:31.050881 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 273/547] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:01:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:31.056969 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e522db3-6917-4477-abbf-be1f94156d99 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:31.172406 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 275/548] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:01:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 12:01:31.256526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e522db3-6917-4477-abbf-be1f94156d99 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:31.281003 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:31.302448 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4eea83-4693-45ad-ba1f-5987776a2110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:31.303053 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4eea83-4693-45ad-ba1f-5987776a2110 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:31.303363 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4eea83-4693-45ad-ba1f-5987776a2110 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:31.303637 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4eea83-4693-45ad-ba1f-5987776a2110 None admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:31.304279 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4eea83-4693-45ad-ba1f-5987776a2110 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:31.304582 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4eea83-4693-45ad-ba1f-5987776a2110 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:31.305945 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4eea83-4693-45ad-ba1f-5987776a2110 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:31.322417 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 274/549] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:01:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:31.329279 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c040cf44-a3e6-4e24-b8d6-b1769788b140 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:31.410217 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:31.511381 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c040cf44-a3e6-4e24-b8d6-b1769788b140 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:31.566281 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da18a08a-c597-4849-8ade-dbdc39b434de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:31.566952 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da18a08a-c597-4849-8ade-dbdc39b434de None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:31.567249 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da18a08a-c597-4849-8ade-dbdc39b434de None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:31.567527 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da18a08a-c597-4849-8ade-dbdc39b434de None admin] PATH_INFO: `/v3/users/swift` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:31.579179 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da18a08a-c597-4849-8ade-dbdc39b434de None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:31.580668 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da18a08a-c597-4849-8ade-dbdc39b434de None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:31.602982 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-da18a08a-c597-4849-8ade-dbdc39b434de None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 26 12:01:31.602982 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:31.602982 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:31.602982 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:31.602982 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:31.602982 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:31.602982 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:31.602982 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:31.602982 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:31.602982 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:31.602982 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:31.602982 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:31.602982 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 12:01:31.602982 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 12:01:31.602982 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 12:01:31.602982 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 12:01:31.604539 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:31.604539 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:31.604539 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 12:01:31.604539 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:01:31.604539 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 12:01:31.604539 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:01:31.604539 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:31.604539 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:31.604539 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:31.604539 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:31.604539 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:31.604539 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:31.604539 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:31.604539 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:31.604539 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:31.604539 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:31.604539 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:31.604539 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:31.605587 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:31.605587 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:31.605587 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 12:01:31.605587 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 12:01:31.605587 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 12:01:31.605587 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 12:01:31.605587 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 12:01:31.605587 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 12:01:31.605587 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 26 12:01:31.605587 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:31.606103 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4eea83-4693-45ad-ba1f-5987776a2110 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:31.608792 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 276/550] 10.222.0.61 () {56 vars in 1096 bytes} [Fri Aug 26 12:01:31 2022] GET /identity/v3/users/swift => generated 83 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:31.616097 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da18a08a-c597-4849-8ade-dbdc39b434de None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:31.751231 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4eea83-4693-45ad-ba1f-5987776a2110 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:31.792261 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504b51a0-d7aa-43ef-a780-d1e376e3cebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:31.792821 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b51a0-d7aa-43ef-a780-d1e376e3cebb None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:31.793168 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b51a0-d7aa-43ef-a780-d1e376e3cebb None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:31.793432 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b51a0-d7aa-43ef-a780-d1e376e3cebb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:31.805019 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504b51a0-d7aa-43ef-a780-d1e376e3cebb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:01:31.807630 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da18a08a-c597-4849-8ade-dbdc39b434de None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:31.811490 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-504b51a0-d7aa-43ef-a780-d1e376e3cebb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 12:01:31.811490 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:31.811490 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:31.811490 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:31.811490 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:31.811490 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:31.811490 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:31.811490 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:31.811490 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:31.811490 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:31.811490 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:31.811490 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:31.811490 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:01:31.811490 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:01:31.811490 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:01:31.811490 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:01:31.812433 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:01:31.812433 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:01:31.812433 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:01:31.812433 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:01:31.812433 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:31.812433 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:31.812433 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:31.812433 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:31.812433 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:31.812433 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:31.812433 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:31.812433 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:31.812433 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:31.812433 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:31.812433 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:31.812433 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:31.813443 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:31.813443 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:31.813443 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:31.813443 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:31.813443 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:01:31.813443 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:01:31.813443 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:01:31.813443 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:01:31.813443 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:01:31.813443 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:01:31.813443 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 12:01:31.813443 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:31.813443 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 275/551] 10.222.0.61 () {56 vars in 1102 bytes} [Fri Aug 26 12:01:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:31.813443 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19789157-e69f-4ece-936a-d0b14aa48fc8 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:31.814200 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19789157-e69f-4ece-936a-d0b14aa48fc8 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:31.814200 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19789157-e69f-4ece-936a-d0b14aa48fc8 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:31.814948 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 276/552] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:01:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:01:31.819113 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165e05e5-fb00-4820-9a1e-702035daffb2 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:31.819113 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165e05e5-fb00-4820-9a1e-702035daffb2 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:31.819113 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165e05e5-fb00-4820-9a1e-702035daffb2 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:31.819656 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 277/553] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:01:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:01:31.820599 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165e05e5-fb00-4820-9a1e-702035daffb2 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:31.822506 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:31.854483 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f42c77a-8526-4ac8-81b9-d20ffbcb27ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:31.855076 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42c77a-8526-4ac8-81b9-d20ffbcb27ef None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:31.855401 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42c77a-8526-4ac8-81b9-d20ffbcb27ef None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:31.855675 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42c77a-8526-4ac8-81b9-d20ffbcb27ef None admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:31.856396 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f42c77a-8526-4ac8-81b9-d20ffbcb27ef None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:31.857220 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f42c77a-8526-4ac8-81b9-d20ffbcb27ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:31.859113 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f42c77a-8526-4ac8-81b9-d20ffbcb27ef None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:31.859605 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:31.882055 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 277/554] 10.222.0.61 () {56 vars in 1135 bytes} [Fri Aug 26 12:01:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:01:31.883843 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fbbb55-8158-44a0-a124-382088259030 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:31.883954 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fbbb55-8158-44a0-a124-382088259030 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:31.884230 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fbbb55-8158-44a0-a124-382088259030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:31.903431 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-f7fbbb55-8158-44a0-a124-382088259030 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:01:31.904811 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-f7fbbb55-8158-44a0-a124-382088259030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:01:31.932803 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165e05e5-fb00-4820-9a1e-702035daffb2 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:31.970624 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5e01db-dad2-4825-afd1-9c5443e00cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:31.971258 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e01db-dad2-4825-afd1-9c5443e00cec None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:31.971539 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e01db-dad2-4825-afd1-9c5443e00cec None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:31.971810 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e01db-dad2-4825-afd1-9c5443e00cec None admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:31.972468 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5e01db-dad2-4825-afd1-9c5443e00cec None admin] RBAC: Adding query filter params (name=service) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:31.972769 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5e01db-dad2-4825-afd1-9c5443e00cec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:31.974123 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5e01db-dad2-4825-afd1-9c5443e00cec None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:31.976991 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 278/555] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:01:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:01:31.978393 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c6f966-4797-4cca-b482-bf70e28b2c5d None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:31.978881 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c6f966-4797-4cca-b482-bf70e28b2c5d None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:31.979765 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c6f966-4797-4cca-b482-bf70e28b2c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:31.982984 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 278/556] 10.222.0.61 () {56 vars in 1103 bytes} [Fri Aug 26 12:01:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:31.983792 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5e01db-dad2-4825-afd1-9c5443e00cec None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:32.007532 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-79c6f966-4797-4cca-b482-bf70e28b2c5d None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:01:32.008795 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-79c6f966-4797-4cca-b482-bf70e28b2c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:01:32.090996 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:32.092556 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 279/557] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:01:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:01:32.095620 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5e01db-dad2-4825-afd1-9c5443e00cec None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:32.100129 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c6f966-4797-4cca-b482-bf70e28b2c5d None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:32.128736 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:32.145060 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefa2c53-1ecb-4067-862c-35a4de403d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:32.145955 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefa2c53-1ecb-4067-862c-35a4de403d90 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:32.146426 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefa2c53-1ecb-4067-862c-35a4de403d90 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:32.146913 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefa2c53-1ecb-4067-862c-35a4de403d90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:32.222578 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c6f966-4797-4cca-b482-bf70e28b2c5d None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:32.229787 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefa2c53-1ecb-4067-862c-35a4de403d90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:32.231424 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefa2c53-1ecb-4067-862c-35a4de403d90 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:32.267285 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaf80a0-fa5c-49a2-9a08-650ba143fd04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:32.267916 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf80a0-fa5c-49a2-9a08-650ba143fd04 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:32.268193 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf80a0-fa5c-49a2-9a08-650ba143fd04 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:32.268464 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf80a0-fa5c-49a2-9a08-650ba143fd04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:32.279217 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaf80a0-fa5c-49a2-9a08-650ba143fd04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:32.280790 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaf80a0-fa5c-49a2-9a08-650ba143fd04 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:32.296459 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-9aaf80a0-fa5c-49a2-9a08-650ba143fd04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:32.296459 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:32.296459 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:32.296459 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:32.296459 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:32.296459 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:32.296459 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:32.296459 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:32.296459 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:32.296459 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:32.296459 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:32.296459 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:32.296459 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:01:32.296459 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:01:32.296459 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:01:32.296459 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:01:32.297367 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:32.297367 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:32.297367 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:32.297367 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:32.297367 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:32.297367 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:32.297367 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:32.297367 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:32.297367 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:32.297367 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:32.297367 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:32.297367 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:32.297367 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:32.297367 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:32.297367 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:32.297367 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:32.297367 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:01:32.298318 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:01:32.298318 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:32.298318 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:32.298318 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 280/558] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:01:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:32.299246 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27278d0c-9898-446b-b53c-6e245179baa1 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:32.299246 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27278d0c-9898-446b-b53c-6e245179baa1 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:32.299419 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27278d0c-9898-446b-b53c-6e245179baa1 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:32.300665 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 281/559] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:01:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:01:32.301248 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27278d0c-9898-446b-b53c-6e245179baa1 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:32.346883 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:32.352155 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 279/560] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:01:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 12:01:32.353044 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefa2c53-1ecb-4067-862c-35a4de403d90 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:32.386105 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:32.411656 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27278d0c-9898-446b-b53c-6e245179baa1 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:32.444957 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb1772-69c8-44b8-a891-73487c6e9286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:32.445576 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb1772-69c8-44b8-a891-73487c6e9286 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:32.445852 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb1772-69c8-44b8-a891-73487c6e9286 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:32.446119 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb1772-69c8-44b8-a891-73487c6e9286 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:32.449843 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefa2c53-1ecb-4067-862c-35a4de403d90 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:32.455983 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fb1772-69c8-44b8-a891-73487c6e9286 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:01:32.461036 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-c8fb1772-69c8-44b8-a891-73487c6e9286 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 12:01:32.461036 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:32.461036 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:32.461036 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:32.461036 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:32.461036 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:32.461036 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:32.461036 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:32.461036 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:32.461036 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:32.461036 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:32.461036 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:32.461036 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:01:32.461036 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:01:32.461036 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:01:32.461036 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:01:32.461927 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:01:32.461927 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:01:32.461927 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:01:32.461927 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:01:32.461927 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:32.461927 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:32.461927 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:32.461927 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:32.461927 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:32.461927 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:32.461927 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:32.461927 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:32.461927 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:32.461927 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:32.461927 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:32.461927 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:32.462753 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:32.462753 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:32.462753 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:32.462753 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:32.462753 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:01:32.462753 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:01:32.462753 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:01:32.462753 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:01:32.462753 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:01:32.462753 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:01:32.462753 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 12:01:32.462753 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:32.467521 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 282/561] 10.222.0.61 () {56 vars in 1102 bytes} [Fri Aug 26 12:01:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:32.468535 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f8244-09c9-4d4f-bedc-ef7b624a27f2 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:32.468837 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f8244-09c9-4d4f-bedc-ef7b624a27f2 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:32.469085 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f8244-09c9-4d4f-bedc-ef7b624a27f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:32.485620 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5f4edd-6867-4e94-b404-e6664c4a33d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:32.486212 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5f4edd-6867-4e94-b404-e6664c4a33d1 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:32.486310 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-028f8244-09c9-4d4f-bedc-ef7b624a27f2 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:01:32.486488 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5f4edd-6867-4e94-b404-e6664c4a33d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:32.486755 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5f4edd-6867-4e94-b404-e6664c4a33d1 None admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:32.487413 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5f4edd-6867-4e94-b404-e6664c4a33d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:32.487507 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-028f8244-09c9-4d4f-bedc-ef7b624a27f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:01:32.487713 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5f4edd-6867-4e94-b404-e6664c4a33d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:32.489101 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5f4edd-6867-4e94-b404-e6664c4a33d1 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:32.499989 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 280/562] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:01:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:32.500767 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5f4edd-6867-4e94-b404-e6664c4a33d1 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:32.552934 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 283/563] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:01:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:01:32.553762 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028f8244-09c9-4d4f-bedc-ef7b624a27f2 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:32.558819 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:32.605576 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5f4edd-6867-4e94-b404-e6664c4a33d1 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:32.625423 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:32.642390 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5b7ad9-8934-48bd-af90-fdc0f57dfa18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:32.643595 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5b7ad9-8934-48bd-af90-fdc0f57dfa18 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:32.643876 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5b7ad9-8934-48bd-af90-fdc0f57dfa18 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:32.644148 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5b7ad9-8934-48bd-af90-fdc0f57dfa18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:32.655707 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5b7ad9-8934-48bd-af90-fdc0f57dfa18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:32.657144 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5b7ad9-8934-48bd-af90-fdc0f57dfa18 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:32.663824 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028f8244-09c9-4d4f-bedc-ef7b624a27f2 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:32.672281 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-9b5b7ad9-8934-48bd-af90-fdc0f57dfa18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:32.672281 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:32.672281 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:32.672281 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:32.672281 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:32.672281 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:32.672281 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:32.672281 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:32.672281 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:32.672281 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:32.672281 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:32.672281 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:32.672281 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:01:32.672281 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:01:32.672281 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:01:32.672281 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:01:32.673215 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:32.673215 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:32.673215 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:32.673215 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:32.673215 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:32.673215 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:32.673215 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:32.673215 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:32.673215 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:32.673215 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:32.673215 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:32.673215 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:32.673215 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:32.673215 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:32.673215 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:32.673215 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:32.673215 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:01:32.674089 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:01:32.674089 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:32.674089 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:32.674089 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 281/564] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:01:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:32.675156 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5b7ad9-8934-48bd-af90-fdc0f57dfa18 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:32.703614 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef0e71d-1aab-4710-bc27-b5e6ed12540f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:32.704395 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef0e71d-1aab-4710-bc27-b5e6ed12540f None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:32.704731 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef0e71d-1aab-4710-bc27-b5e6ed12540f None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:32.705314 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef0e71d-1aab-4710-bc27-b5e6ed12540f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:32.717218 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef0e71d-1aab-4710-bc27-b5e6ed12540f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:01:32.722089 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-0ef0e71d-1aab-4710-bc27-b5e6ed12540f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 12:01:32.722089 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:32.722089 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:32.722089 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:32.722089 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:32.722089 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:32.722089 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:32.722089 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:32.722089 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:32.722089 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:32.722089 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:32.722089 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:32.722089 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:01:32.722089 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:01:32.722089 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:01:32.722089 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:01:32.723238 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:01:32.723238 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:01:32.723238 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:01:32.723238 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:01:32.723238 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:32.723238 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:32.723238 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:32.723238 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:32.723238 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:32.723238 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:32.723238 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:32.723238 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:32.723238 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:32.723238 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:32.723238 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:32.723238 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:32.724449 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:32.724449 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:32.724449 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:32.724449 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:32.724449 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:01:32.724449 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:01:32.724449 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:01:32.724449 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:01:32.724449 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:01:32.724449 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:01:32.724449 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 12:01:32.724449 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:32.724449 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 284/565] 10.222.0.61 () {56 vars in 1102 bytes} [Fri Aug 26 12:01:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:32.725224 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef0e71d-1aab-4710-bc27-b5e6ed12540f None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:32.795324 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5b7ad9-8934-48bd-af90-fdc0f57dfa18 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:32.806738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:32.831573 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef0e71d-1aab-4710-bc27-b5e6ed12540f None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:32.833877 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ddba06-e4e9-484e-bf91-f4e3d2254900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:32.834487 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ddba06-e4e9-484e-bf91-f4e3d2254900 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:32.834767 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ddba06-e4e9-484e-bf91-f4e3d2254900 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:32.835067 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ddba06-e4e9-484e-bf91-f4e3d2254900 None admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:32.835767 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ddba06-e4e9-484e-bf91-f4e3d2254900 None admin] RBAC: Adding query filter params (name=service) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:32.836070 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ddba06-e4e9-484e-bf91-f4e3d2254900 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:32.837616 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ddba06-e4e9-484e-bf91-f4e3d2254900 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:32.846191 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 282/566] 10.222.0.61 () {56 vars in 1103 bytes} [Fri Aug 26 12:01:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:32.847049 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ddba06-e4e9-484e-bf91-f4e3d2254900 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:32.862865 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:32.873862 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7458a1-5dcd-48c1-92a2-40fa6d823009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:32.874492 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7458a1-5dcd-48c1-92a2-40fa6d823009 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:32.874775 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7458a1-5dcd-48c1-92a2-40fa6d823009 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:32.875053 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7458a1-5dcd-48c1-92a2-40fa6d823009 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:32.881986 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7458a1-5dcd-48c1-92a2-40fa6d823009 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:32.883724 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7458a1-5dcd-48c1-92a2-40fa6d823009 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:32.895265 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-4f7458a1-5dcd-48c1-92a2-40fa6d823009 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 12:01:32.895265 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:32.895265 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:32.895265 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:32.895265 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:32.895265 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:32.895265 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:32.895265 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:32.895265 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:32.895265 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:32.895265 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:32.895265 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:32.895265 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 12:01:32.895265 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 12:01:32.895265 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 12:01:32.895265 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 12:01:32.896342 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:32.896342 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:32.896342 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 12:01:32.896342 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:01:32.896342 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 12:01:32.896342 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:01:32.896342 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:32.896342 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:32.896342 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:32.896342 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:32.896342 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:32.896342 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:32.896342 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:32.896342 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:32.896342 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:32.896342 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:32.896342 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:32.896342 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:32.897474 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:32.897474 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:32.897474 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 12:01:32.897474 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 12:01:32.897474 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 12:01:32.897474 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 12:01:32.897474 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 12:01:32.897474 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 12:01:32.897474 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 12:01:32.897474 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:32.897474 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 285/567] 10.222.0.61 () {56 vars in 1093 bytes} [Fri Aug 26 12:01:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:32.897474 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7458a1-5dcd-48c1-92a2-40fa6d823009 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:32.948416 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ddba06-e4e9-484e-bf91-f4e3d2254900 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:32.982328 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c39d71-681a-4074-a5ad-26dd02dfd132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:32.982925 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c39d71-681a-4074-a5ad-26dd02dfd132 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:32.983234 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c39d71-681a-4074-a5ad-26dd02dfd132 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:32.983506 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c39d71-681a-4074-a5ad-26dd02dfd132 None admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:32.984144 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c39d71-681a-4074-a5ad-26dd02dfd132 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:32.984444 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c39d71-681a-4074-a5ad-26dd02dfd132 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:32.985821 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c39d71-681a-4074-a5ad-26dd02dfd132 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:32.995541 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7458a1-5dcd-48c1-92a2-40fa6d823009 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:32.997187 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 283/568] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:01:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:32.998596 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c39d71-681a-4074-a5ad-26dd02dfd132 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:33.030378 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c51d47-76c4-4165-ba48-489887cb805c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:33.031098 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c51d47-76c4-4165-ba48-489887cb805c None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:33.031419 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c51d47-76c4-4165-ba48-489887cb805c None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:33.031574 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:33.031675 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c51d47-76c4-4165-ba48-489887cb805c None admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:33.032345 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c51d47-76c4-4165-ba48-489887cb805c None admin] RBAC: Adding query filter params (name=service) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:33.032851 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c51d47-76c4-4165-ba48-489887cb805c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:33.034200 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c51d47-76c4-4165-ba48-489887cb805c None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:33.042428 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 286/569] 10.222.0.61 () {56 vars in 1103 bytes} [Fri Aug 26 12:01:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:33.043148 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c51d47-76c4-4165-ba48-489887cb805c None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:33.090531 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:33.105320 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c39d71-681a-4074-a5ad-26dd02dfd132 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:33.140443 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3378a1a5-909e-4949-be5e-7713c78da068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:33.141001 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3378a1a5-909e-4949-be5e-7713c78da068 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:33.141279 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3378a1a5-909e-4949-be5e-7713c78da068 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:33.141548 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3378a1a5-909e-4949-be5e-7713c78da068 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:33.142638 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c51d47-76c4-4165-ba48-489887cb805c None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:33.151704 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3378a1a5-909e-4949-be5e-7713c78da068 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:01:33.157106 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-3378a1a5-909e-4949-be5e-7713c78da068 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 12:01:33.157106 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:33.157106 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:33.157106 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:33.157106 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:33.157106 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:33.157106 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:33.157106 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:33.157106 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:33.157106 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:33.157106 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:33.157106 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:33.157106 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:01:33.157106 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:01:33.157106 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:01:33.157106 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:01:33.158114 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:01:33.158114 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:01:33.158114 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:01:33.158114 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:01:33.158114 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:33.158114 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:33.158114 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:33.158114 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:33.158114 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:33.158114 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:33.158114 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:33.158114 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:33.158114 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:33.158114 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:33.158114 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:33.158114 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:33.158975 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:33.158975 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:33.158975 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:33.158975 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:33.158975 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:01:33.158975 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:01:33.158975 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:01:33.158975 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:01:33.158975 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:01:33.158975 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:01:33.158975 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 12:01:33.158975 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:33.158975 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 284/570] 10.222.0.61 () {56 vars in 1102 bytes} [Fri Aug 26 12:01:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:33.158975 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3378a1a5-909e-4949-be5e-7713c78da068 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:33.180223 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41656dc-b2c6-4c61-a23e-344d2c4cf277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:33.180850 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41656dc-b2c6-4c61-a23e-344d2c4cf277 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:33.181128 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41656dc-b2c6-4c61-a23e-344d2c4cf277 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:33.181399 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41656dc-b2c6-4c61-a23e-344d2c4cf277 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:33.191963 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41656dc-b2c6-4c61-a23e-344d2c4cf277 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:33.194298 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41656dc-b2c6-4c61-a23e-344d2c4cf277 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:33.209340 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-f41656dc-b2c6-4c61-a23e-344d2c4cf277 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:33.209340 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:33.209340 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:33.209340 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:33.209340 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:33.209340 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:33.209340 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:33.209340 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:33.209340 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:33.209340 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:33.209340 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:33.209340 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:33.209340 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:01:33.209340 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:01:33.209340 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:01:33.209340 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:01:33.210470 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:33.210470 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:33.210470 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:33.210470 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:33.210470 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:33.210470 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:33.210470 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:33.210470 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:33.210470 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:33.210470 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:33.210470 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:33.210470 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:33.210470 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:33.210470 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:33.210470 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:33.210470 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:33.210470 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:01:33.211547 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:01:33.211547 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:33.211547 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:33.215153 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 287/571] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:01:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:33.216562 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41656dc-b2c6-4c61-a23e-344d2c4cf277 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:33.250313 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:33.259873 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3378a1a5-909e-4949-be5e-7713c78da068 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:33.292966 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ca26d4-546a-430e-9c8f-b42fa36d21df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:33.293563 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ca26d4-546a-430e-9c8f-b42fa36d21df None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:33.293849 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ca26d4-546a-430e-9c8f-b42fa36d21df None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:33.294194 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ca26d4-546a-430e-9c8f-b42fa36d21df None admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:33.294915 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ca26d4-546a-430e-9c8f-b42fa36d21df None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:33.295235 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ca26d4-546a-430e-9c8f-b42fa36d21df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:33.296681 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ca26d4-546a-430e-9c8f-b42fa36d21df None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:33.315188 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:33.318498 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41656dc-b2c6-4c61-a23e-344d2c4cf277 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:33.332904 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 285/572] 10.222.0.61 () {56 vars in 1133 bytes} [Fri Aug 26 12:01:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:33.333743 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ca26d4-546a-430e-9c8f-b42fa36d21df None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:33.353122 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa5528e-7b78-457a-86e3-fbdb285cb466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:33.353746 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa5528e-7b78-457a-86e3-fbdb285cb466 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:33.354027 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa5528e-7b78-457a-86e3-fbdb285cb466 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:33.354303 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa5528e-7b78-457a-86e3-fbdb285cb466 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:33.361406 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa5528e-7b78-457a-86e3-fbdb285cb466 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:33.362844 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa5528e-7b78-457a-86e3-fbdb285cb466 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:33.374364 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-caa5528e-7b78-457a-86e3-fbdb285cb466 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:01:33.374364 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:33.374364 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:33.374364 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:33.374364 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:33.374364 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:33.374364 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:33.374364 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:33.374364 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:33.374364 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:33.374364 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:33.374364 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:33.374364 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 12:01:33.374364 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 12:01:33.374364 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 12:01:33.375380 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 12:01:33.375380 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:33.375380 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:33.375380 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:33.375380 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:33.375380 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:33.375380 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:33.375380 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:33.375380 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:33.375380 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:33.375380 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:33.375380 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:33.375380 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:33.375380 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:33.375380 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:33.375380 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:33.375380 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:33.376298 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 12:01:33.376298 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 12:01:33.376298 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 12:01:33.376298 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 12:01:33.376298 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 12:01:33.376298 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 12:01:33.376298 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:01:33.376298 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:33.376298 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 288/573] 10.222.0.61 () {56 vars in 1111 bytes} [Fri Aug 26 12:01:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:33.376787 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa5528e-7b78-457a-86e3-fbdb285cb466 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:33.439267 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ca26d4-546a-430e-9c8f-b42fa36d21df None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:33.473085 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e523c00-fe31-448e-a67f-a1f65e9de88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:33.473729 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e523c00-fe31-448e-a67f-a1f65e9de88a None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:33.474007 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e523c00-fe31-448e-a67f-a1f65e9de88a None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:33.474275 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e523c00-fe31-448e-a67f-a1f65e9de88a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:33.474382 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa5528e-7b78-457a-86e3-fbdb285cb466 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:33.484762 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e523c00-fe31-448e-a67f-a1f65e9de88a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:33.486198 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e523c00-fe31-448e-a67f-a1f65e9de88a None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:33.501097 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-4e523c00-fe31-448e-a67f-a1f65e9de88a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:33.501097 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:33.501097 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:33.501097 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:33.501097 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:33.501097 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:33.501097 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:33.501097 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:33.501097 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:33.501097 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:33.501097 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:33.501097 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:33.501097 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:01:33.501097 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:01:33.501097 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:01:33.501097 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:01:33.506952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:33.506952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:33.506952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:33.506952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:33.506952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:33.506952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:33.506952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:33.506952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:33.506952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:33.506952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:33.506952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:33.506952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:33.506952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:33.506952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:33.506952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:33.506952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:33.506952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:01:33.508802 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:01:33.508802 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:33.508802 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:33.508802 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 286/574] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:01:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:33.508802 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e523c00-fe31-448e-a67f-a1f65e9de88a None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:33.508802 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:33.509479 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5861fb3e-ff54-4f48-a623-3984dc173e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:33.510090 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861fb3e-ff54-4f48-a623-3984dc173e06 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:33.510456 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861fb3e-ff54-4f48-a623-3984dc173e06 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:33.510727 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861fb3e-ff54-4f48-a623-3984dc173e06 None admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:33.511412 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5861fb3e-ff54-4f48-a623-3984dc173e06 None admin] RBAC: Adding query filter params (name=service) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:33.511714 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5861fb3e-ff54-4f48-a623-3984dc173e06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:33.513188 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5861fb3e-ff54-4f48-a623-3984dc173e06 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:33.526155 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 289/575] 10.222.0.61 () {56 vars in 1103 bytes} [Fri Aug 26 12:01:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:33.526155 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5861fb3e-ff54-4f48-a623-3984dc173e06 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:33.546036 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:33.616877 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e523c00-fe31-448e-a67f-a1f65e9de88a None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:33.628181 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5861fb3e-ff54-4f48-a623-3984dc173e06 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:33.650455 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b917c4b-da8e-4ea6-b799-3590671d6c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:33.651056 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b917c4b-da8e-4ea6-b799-3590671d6c3d None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:33.651359 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b917c4b-da8e-4ea6-b799-3590671d6c3d None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:33.651631 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b917c4b-da8e-4ea6-b799-3590671d6c3d None admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:33.652261 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b917c4b-da8e-4ea6-b799-3590671d6c3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:33.652557 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b917c4b-da8e-4ea6-b799-3590671d6c3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:33.654117 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b917c4b-da8e-4ea6-b799-3590671d6c3d None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:33.661958 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f388cf6-6daa-4d56-ad3a-3460870bd6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:33.662575 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f388cf6-6daa-4d56-ad3a-3460870bd6d0 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:33.662853 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f388cf6-6daa-4d56-ad3a-3460870bd6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:33.663124 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f388cf6-6daa-4d56-ad3a-3460870bd6d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:33.666436 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 287/576] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:01:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:33.667236 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b917c4b-da8e-4ea6-b799-3590671d6c3d None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:33.675749 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f388cf6-6daa-4d56-ad3a-3460870bd6d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:33.677194 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f388cf6-6daa-4d56-ad3a-3460870bd6d0 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:33.691821 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-1f388cf6-6daa-4d56-ad3a-3460870bd6d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:33.691821 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:33.691821 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:33.691821 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:33.691821 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:33.691821 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:33.691821 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:33.691821 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:33.691821 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:33.691821 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:33.691821 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:33.691821 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:33.691821 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:01:33.691821 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:01:33.691821 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:01:33.691821 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:01:33.692730 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:33.692730 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:33.692730 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:33.692730 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:33.692730 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:33.692730 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:33.692730 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:33.692730 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:33.692730 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:33.692730 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:33.692730 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:33.692730 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:33.692730 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:33.692730 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:33.692730 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:33.692730 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:33.692730 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:01:33.693693 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:01:33.693693 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:33.693693 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:33.694122 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 290/577] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:01:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:33.695012 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f388cf6-6daa-4d56-ad3a-3460870bd6d0 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:33.732150 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:33.769651 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:33.773540 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b917c4b-da8e-4ea6-b799-3590671d6c3d None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:33.796012 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f388cf6-6daa-4d56-ad3a-3460870bd6d0 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:33.806731 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8964bc83-a72b-4cbc-a24a-ea1dbfb7e572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:33.807354 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8964bc83-a72b-4cbc-a24a-ea1dbfb7e572 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:33.807636 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8964bc83-a72b-4cbc-a24a-ea1dbfb7e572 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:33.807903 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8964bc83-a72b-4cbc-a24a-ea1dbfb7e572 None admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:33.808581 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8964bc83-a72b-4cbc-a24a-ea1dbfb7e572 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:33.808878 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8964bc83-a72b-4cbc-a24a-ea1dbfb7e572 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:33.810472 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8964bc83-a72b-4cbc-a24a-ea1dbfb7e572 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:33.821660 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 288/578] 10.222.0.61 () {56 vars in 1148 bytes} [Fri Aug 26 12:01:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:33.822246 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8964bc83-a72b-4cbc-a24a-ea1dbfb7e572 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:33.830005 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd62cf4-22e5-42ad-82de-f24a9ec2b58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:33.830610 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd62cf4-22e5-42ad-82de-f24a9ec2b58e None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:33.830894 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd62cf4-22e5-42ad-82de-f24a9ec2b58e None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:33.831164 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd62cf4-22e5-42ad-82de-f24a9ec2b58e None admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:33.831842 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd62cf4-22e5-42ad-82de-f24a9ec2b58e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:33.832145 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd62cf4-22e5-42ad-82de-f24a9ec2b58e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:33.833494 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd62cf4-22e5-42ad-82de-f24a9ec2b58e None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:33.849874 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 291/579] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:01:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:33.849874 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd62cf4-22e5-42ad-82de-f24a9ec2b58e None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:33.921768 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8964bc83-a72b-4cbc-a24a-ea1dbfb7e572 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:33.945058 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:33.948256 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd62cf4-22e5-42ad-82de-f24a9ec2b58e None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:33.956919 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fab16fa-a4e9-4ea3-b6ea-f774f06b1884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:33.957539 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fab16fa-a4e9-4ea3-b6ea-f774f06b1884 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:33.957814 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fab16fa-a4e9-4ea3-b6ea-f774f06b1884 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:33.958083 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fab16fa-a4e9-4ea3-b6ea-f774f06b1884 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:33.968911 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fab16fa-a4e9-4ea3-b6ea-f774f06b1884 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:33.970342 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fab16fa-a4e9-4ea3-b6ea-f774f06b1884 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:33.983904 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cec685f-49e8-40f3-878e-7dc5575df75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:33.984519 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-4fab16fa-a4e9-4ea3-b6ea-f774f06b1884 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:33.984519 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:33.984519 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:33.984519 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:33.984519 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:33.984519 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:33.984519 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:33.984519 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:33.984519 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:33.984519 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:33.984519 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:33.984519 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:33.984519 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:01:33.984519 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:01:33.984519 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:01:33.984519 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:01:33.985591 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:33.985591 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:33.985591 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:33.985591 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:33.985591 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:33.985591 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:33.985591 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:33.985591 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:33.985591 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:33.985591 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:33.985591 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:33.985591 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:33.985591 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:33.985591 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:33.985591 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:33.985591 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:33.985591 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:01:33.986605 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:01:33.986605 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:33.986605 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:33.986802 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec685f-49e8-40f3-878e-7dc5575df75e None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:33.986802 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec685f-49e8-40f3-878e-7dc5575df75e None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:33.986802 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cec685f-49e8-40f3-878e-7dc5575df75e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:33.987025 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 289/580] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:01:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:33.987025 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fab16fa-a4e9-4ea3-b6ea-f774f06b1884 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:33.988686 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:33.996101 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cec685f-49e8-40f3-878e-7dc5575df75e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:33.997574 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cec685f-49e8-40f3-878e-7dc5575df75e None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:34.009382 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-2cec685f-49e8-40f3-878e-7dc5575df75e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 26 12:01:34.009382 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:34.009382 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:34.009382 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:34.009382 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:34.009382 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:34.009382 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:34.009382 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:34.009382 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:34.009382 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:34.009382 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:34.009382 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:34.009382 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 12:01:34.009382 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 12:01:34.009382 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 12:01:34.009382 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 12:01:34.010510 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:34.010510 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:34.010510 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 12:01:34.010510 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:01:34.010510 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 12:01:34.010510 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:01:34.010510 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:34.010510 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:34.010510 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:34.010510 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:34.010510 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:34.010510 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:34.010510 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:34.010510 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:34.010510 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:34.010510 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:34.010510 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:34.010510 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:34.011713 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:34.011713 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:34.011713 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 12:01:34.011713 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 12:01:34.011713 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 12:01:34.011713 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 12:01:34.011713 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 12:01:34.011713 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 12:01:34.011713 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 26 12:01:34.011713 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:34.011713 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 292/581] 10.222.0.61 () {56 vars in 1102 bytes} [Fri Aug 26 12:01:33 2022] GET /identity/v3/users/neutron => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:34.011713 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cec685f-49e8-40f3-878e-7dc5575df75e None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:34.087399 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fab16fa-a4e9-4ea3-b6ea-f774f06b1884 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:34.112398 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cec685f-49e8-40f3-878e-7dc5575df75e None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:34.119549 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9b0280-3918-4e7a-842d-16c84a9fd7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:34.120197 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9b0280-3918-4e7a-842d-16c84a9fd7e2 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:34.120477 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9b0280-3918-4e7a-842d-16c84a9fd7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:34.120749 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9b0280-3918-4e7a-842d-16c84a9fd7e2 None admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:34.121378 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9b0280-3918-4e7a-842d-16c84a9fd7e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:34.121675 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9b0280-3918-4e7a-842d-16c84a9fd7e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:34.123753 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9b0280-3918-4e7a-842d-16c84a9fd7e2 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:34.135360 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 290/582] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:01:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:34.136149 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9b0280-3918-4e7a-842d-16c84a9fd7e2 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:34.152139 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc03276-74a0-45df-b335-1d5b7f345f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:34.152741 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc03276-74a0-45df-b335-1d5b7f345f55 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:34.153023 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc03276-74a0-45df-b335-1d5b7f345f55 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:34.153297 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc03276-74a0-45df-b335-1d5b7f345f55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:34.154076 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc03276-74a0-45df-b335-1d5b7f345f55 None admin] RBAC: Adding query filter params (role.id=519347d2bd2c4dc1aadb0b9f8c3c8e3d, scope.project.id=ecd68423a83a427681d44dbf266fce6e, user.id=ead95a244d11432d9fe9f718e1a75f38) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:34.154383 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc03276-74a0-45df-b335-1d5b7f345f55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:34.155849 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc03276-74a0-45df-b335-1d5b7f345f55 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:34.161092 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 293/583] 10.222.0.61 () {56 vars in 1374 bytes} [Fri Aug 26 12:01:34 2022] GET /identity/v3/role_assignments?user.id=ead95a244d11432d9fe9f718e1a75f38&scope.project.id=ecd68423a83a427681d44dbf266fce6e&role.id=519347d2bd2c4dc1aadb0b9f8c3c8e3d => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:34.161857 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc03276-74a0-45df-b335-1d5b7f345f55 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:34.168438 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:34.244162 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:34.252685 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9b0280-3918-4e7a-842d-16c84a9fd7e2 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:34.292240 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc03276-74a0-45df-b335-1d5b7f345f55 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:34.293431 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bea0c6-8dca-4080-a70e-3311f7a1186d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:34.294090 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bea0c6-8dca-4080-a70e-3311f7a1186d None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:34.294353 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bea0c6-8dca-4080-a70e-3311f7a1186d None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:34.294804 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bea0c6-8dca-4080-a70e-3311f7a1186d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:34.303590 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bea0c6-8dca-4080-a70e-3311f7a1186d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:34.305272 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bea0c6-8dca-4080-a70e-3311f7a1186d None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:34.319391 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-e5bea0c6-8dca-4080-a70e-3311f7a1186d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 26 12:01:34.319391 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:34.319391 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:34.319391 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:34.319391 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:34.319391 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:34.319391 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:34.319391 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:34.319391 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:34.319391 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:34.319391 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:34.319391 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:34.319391 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 12:01:34.319391 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 12:01:34.319391 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 12:01:34.319391 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 12:01:34.320367 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:34.320367 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:34.320367 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 12:01:34.320367 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:01:34.320367 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 12:01:34.320367 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:01:34.320367 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:34.320367 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:34.320367 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:34.320367 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:34.320367 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:34.320367 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:34.320367 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:34.320367 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:34.320367 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:34.320367 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:34.320367 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:34.320367 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:34.321349 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:34.321349 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:34.321349 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 12:01:34.321349 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 12:01:34.321349 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 12:01:34.321349 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 12:01:34.321349 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 12:01:34.321349 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 12:01:34.321349 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 26 12:01:34.321349 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:34.321349 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 291/584] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:01:34 2022] GET /identity/v3/users/cinder => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:34.321349 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bea0c6-8dca-4080-a70e-3311f7a1186d None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:34.340161 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b53f39-e760-49df-a772-cd571a62383d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:34.342265 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b53f39-e760-49df-a772-cd571a62383d None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:34.342725 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b53f39-e760-49df-a772-cd571a62383d None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:34.343337 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b53f39-e760-49df-a772-cd571a62383d None admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:34.344450 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b53f39-e760-49df-a772-cd571a62383d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:34.344928 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b53f39-e760-49df-a772-cd571a62383d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:34.346475 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b53f39-e760-49df-a772-cd571a62383d None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:34.363833 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 294/585] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:01:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:34.363833 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b53f39-e760-49df-a772-cd571a62383d None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:34.430109 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:34.438764 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bea0c6-8dca-4080-a70e-3311f7a1186d None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:34.468740 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b53f39-e760-49df-a772-cd571a62383d None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:34.474921 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3271a60c-2484-49d7-a14b-dbb45af8e120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:34.475716 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3271a60c-2484-49d7-a14b-dbb45af8e120 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:34.476171 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3271a60c-2484-49d7-a14b-dbb45af8e120 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:34.476437 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3271a60c-2484-49d7-a14b-dbb45af8e120 None admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:34.477165 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3271a60c-2484-49d7-a14b-dbb45af8e120 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:34.477466 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3271a60c-2484-49d7-a14b-dbb45af8e120 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:34.478893 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3271a60c-2484-49d7-a14b-dbb45af8e120 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:34.489329 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:34.501435 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 292/586] 10.222.0.61 () {56 vars in 1139 bytes} [Fri Aug 26 12:01:34 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:34.501435 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3271a60c-2484-49d7-a14b-dbb45af8e120 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:34.512429 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6b6172-1b22-4592-a55b-8e11a186be56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:34.513729 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b6172-1b22-4592-a55b-8e11a186be56 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:34.514288 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b6172-1b22-4592-a55b-8e11a186be56 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:34.514627 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b6172-1b22-4592-a55b-8e11a186be56 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:34.521531 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6b6172-1b22-4592-a55b-8e11a186be56 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:34.523129 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6b6172-1b22-4592-a55b-8e11a186be56 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:34.535417 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-4f6b6172-1b22-4592-a55b-8e11a186be56 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:01:34.535417 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:34.535417 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:34.535417 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:34.535417 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:34.535417 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:34.535417 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:34.535417 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:34.535417 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:34.535417 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:34.535417 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:34.535417 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:34.535417 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 12:01:34.535417 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 12:01:34.535417 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 12:01:34.536327 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 12:01:34.536327 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:34.536327 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:34.536327 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:34.536327 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:34.536327 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:34.536327 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:34.536327 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:34.536327 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:34.536327 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:34.536327 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:34.536327 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:34.536327 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:34.536327 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:34.536327 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:34.536327 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:34.536327 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:34.537210 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 12:01:34.537210 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 12:01:34.537210 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 12:01:34.537210 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 12:01:34.537210 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 12:01:34.537210 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 12:01:34.537210 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:01:34.537210 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:34.537210 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 295/587] 10.222.0.61 () {56 vars in 1111 bytes} [Fri Aug 26 12:01:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:34.537700 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6b6172-1b22-4592-a55b-8e11a186be56 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:34.616845 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3271a60c-2484-49d7-a14b-dbb45af8e120 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:34.651754 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6b6172-1b22-4592-a55b-8e11a186be56 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:34.654518 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d3a875-672e-44ac-8486-db036245f66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:34.655127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d3a875-672e-44ac-8486-db036245f66c None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:34.655445 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d3a875-672e-44ac-8486-db036245f66c None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:34.655719 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d3a875-672e-44ac-8486-db036245f66c None admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:34.656441 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d3a875-672e-44ac-8486-db036245f66c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:34.656743 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d3a875-672e-44ac-8486-db036245f66c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:34.658194 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d3a875-672e-44ac-8486-db036245f66c None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:34.676826 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 293/588] 10.222.0.61 () {56 vars in 1137 bytes} [Fri Aug 26 12:01:34 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:34.677648 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d3a875-672e-44ac-8486-db036245f66c None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:34.691266 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:34.696121 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba529e2-c990-46b5-b3b5-5c8785becf30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:34.696882 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba529e2-c990-46b5-b3b5-5c8785becf30 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:34.697165 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba529e2-c990-46b5-b3b5-5c8785becf30 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:34.697437 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba529e2-c990-46b5-b3b5-5c8785becf30 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:34.704853 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba529e2-c990-46b5-b3b5-5c8785becf30 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:34.706284 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba529e2-c990-46b5-b3b5-5c8785becf30 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:34.718169 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-eba529e2-c990-46b5-b3b5-5c8785becf30 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 26 12:01:34.718169 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:34.718169 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:34.718169 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:34.718169 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:34.718169 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:34.718169 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:34.718169 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:34.718169 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:34.718169 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:34.718169 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:34.718169 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:34.718169 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 12:01:34.718169 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 12:01:34.718169 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 12:01:34.718169 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 12:01:34.719328 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:34.719328 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:34.719328 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 12:01:34.719328 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:01:34.719328 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 12:01:34.719328 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:01:34.719328 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:34.719328 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:34.719328 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:34.719328 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:34.719328 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:34.719328 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:34.719328 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:34.719328 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:34.719328 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:34.719328 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:34.719328 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:34.719328 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:34.720349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:34.720349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:34.720349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 12:01:34.720349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 12:01:34.720349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 12:01:34.720349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 12:01:34.720349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 12:01:34.720349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 12:01:34.720349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 26 12:01:34.720349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:34.720349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 296/589] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:01:34 2022] GET /identity/v3/users/glance => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:34.720349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba529e2-c990-46b5-b3b5-5c8785becf30 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:34.767878 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:34.796732 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d3a875-672e-44ac-8486-db036245f66c None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:34.832370 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c923401-df39-4bae-8728-fc7b1fb5f44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:34.833136 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c923401-df39-4bae-8728-fc7b1fb5f44c None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:34.833535 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c923401-df39-4bae-8728-fc7b1fb5f44c None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:34.833901 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba529e2-c990-46b5-b3b5-5c8785becf30 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:34.834002 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c923401-df39-4bae-8728-fc7b1fb5f44c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:34.845829 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c923401-df39-4bae-8728-fc7b1fb5f44c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:34.847409 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c923401-df39-4bae-8728-fc7b1fb5f44c None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:34.862660 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-5c923401-df39-4bae-8728-fc7b1fb5f44c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:34.862660 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:34.862660 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:34.862660 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:34.862660 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:34.862660 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:34.862660 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:34.862660 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:34.862660 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:34.862660 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:34.862660 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:34.862660 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:34.862660 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:01:34.862660 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:01:34.862660 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:01:34.862660 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:01:34.863806 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:34.863806 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:34.863806 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:34.863806 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:34.863806 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:34.863806 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:34.863806 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:34.863806 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:34.863806 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:34.863806 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:34.863806 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:34.863806 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:34.863806 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:34.863806 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:34.863806 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:34.863806 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:34.863806 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:01:34.864916 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:01:34.864916 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:34.864916 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:34.865426 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 294/590] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:01:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:34.866257 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c923401-df39-4bae-8728-fc7b1fb5f44c None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:34.877573 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52519e21-3403-4904-b78b-f1d4d8b1052c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:34.878177 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52519e21-3403-4904-b78b-f1d4d8b1052c None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:34.878458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52519e21-3403-4904-b78b-f1d4d8b1052c None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:34.878732 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52519e21-3403-4904-b78b-f1d4d8b1052c None admin] PATH_INFO: `/v3/projects` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:34.879452 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52519e21-3403-4904-b78b-f1d4d8b1052c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:34.879760 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52519e21-3403-4904-b78b-f1d4d8b1052c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:34.881182 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52519e21-3403-4904-b78b-f1d4d8b1052c None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:34.894132 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 297/591] 10.222.0.61 () {56 vars in 1148 bytes} [Fri Aug 26 12:01:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:34.894923 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52519e21-3403-4904-b78b-f1d4d8b1052c None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:34.936934 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:34.979032 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c923401-df39-4bae-8728-fc7b1fb5f44c None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:34.994374 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:34.999039 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52519e21-3403-4904-b78b-f1d4d8b1052c None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:35.018484 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d3cc84-d1d5-4e1d-87e7-66f527300ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:35.019143 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d3cc84-d1d5-4e1d-87e7-66f527300ca5 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:35.019509 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d3cc84-d1d5-4e1d-87e7-66f527300ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:35.019847 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d3cc84-d1d5-4e1d-87e7-66f527300ca5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:35.031010 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d3cc84-d1d5-4e1d-87e7-66f527300ca5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:35.032621 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d3cc84-d1d5-4e1d-87e7-66f527300ca5 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:35.038962 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52b1315-3e4a-48d6-9d57-769c1e028fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:35.040064 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b1315-3e4a-48d6-9d57-769c1e028fd8 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:35.040422 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b1315-3e4a-48d6-9d57-769c1e028fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:35.040767 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b1315-3e4a-48d6-9d57-769c1e028fd8 None admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:35.041947 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52b1315-3e4a-48d6-9d57-769c1e028fd8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:35.042328 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52b1315-3e4a-48d6-9d57-769c1e028fd8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:35.043849 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52b1315-3e4a-48d6-9d57-769c1e028fd8 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:35.047725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-11d3cc84-d1d5-4e1d-87e7-66f527300ca5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:35.047725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:35.047725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:35.047725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:35.047725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:35.047725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:35.047725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:35.047725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:35.047725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:35.047725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:35.047725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:35.047725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:35.047725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:01:35.047725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:01:35.047725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:01:35.047725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:01:35.048862 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:35.048862 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:35.048862 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:35.048862 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:35.048862 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:35.048862 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:35.048862 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:35.048862 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:35.048862 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:35.048862 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:35.048862 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:35.048862 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:35.048862 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:35.048862 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:35.048862 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:35.048862 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:35.048862 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:01:35.049984 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:01:35.049984 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:35.049984 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:35.050595 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 295/592] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:01:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:35.051516 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d3cc84-d1d5-4e1d-87e7-66f527300ca5 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:35.072561 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 298/593] 10.222.0.61 () {56 vars in 1137 bytes} [Fri Aug 26 12:01:34 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:35.072561 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52b1315-3e4a-48d6-9d57-769c1e028fd8 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:35.166701 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d3cc84-d1d5-4e1d-87e7-66f527300ca5 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:35.180283 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52b1315-3e4a-48d6-9d57-769c1e028fd8 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:35.181520 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:35.204447 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0733339-a1d6-4217-a822-dc3c855948f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:35.205050 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0733339-a1d6-4217-a822-dc3c855948f0 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:35.205325 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0733339-a1d6-4217-a822-dc3c855948f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:35.205594 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0733339-a1d6-4217-a822-dc3c855948f0 None admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:35.206229 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0733339-a1d6-4217-a822-dc3c855948f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:35.206525 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0733339-a1d6-4217-a822-dc3c855948f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:35.207922 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0733339-a1d6-4217-a822-dc3c855948f0 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:35.217437 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff52811-975a-4a1d-b27e-6047086bd9ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:35.218093 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff52811-975a-4a1d-b27e-6047086bd9ec None admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:35.218376 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff52811-975a-4a1d-b27e-6047086bd9ec None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:35.218647 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff52811-975a-4a1d-b27e-6047086bd9ec None admin] PATH_INFO: `/v3/projects/ecd68423a83a427681d44dbf266fce6e/users/7aacd7d6d17e4f10ba5c9107a12beafa/roles/519347d2bd2c4dc1aadb0b9f8c3c8e3d` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:35.223023 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 296/594] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:01:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:35.224161 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0733339-a1d6-4217-a822-dc3c855948f0 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:35.235071 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff52811-975a-4a1d-b27e-6047086bd9ec None admin] RBAC: Authorizing `identity:create_grant(project_id=ecd68423a83a427681d44dbf266fce6e, user_id=7aacd7d6d17e4f10ba5c9107a12beafa, role_id=519347d2bd2c4dc1aadb0b9f8c3c8e3d)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:35.236575 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff52811-975a-4a1d-b27e-6047086bd9ec None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:35.253022 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 299/595] 10.222.0.61 () {58 vars in 1439 bytes} [Fri Aug 26 12:01:35 2022] PUT /identity/v3/projects/ecd68423a83a427681d44dbf266fce6e/users/7aacd7d6d17e4f10ba5c9107a12beafa/roles/519347d2bd2c4dc1aadb0b9f8c3c8e3d => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:01:35.254315 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff52811-975a-4a1d-b27e-6047086bd9ec None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:35.272008 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:35.351508 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0733339-a1d6-4217-a822-dc3c855948f0 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:35.391021 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb6165d-9437-4c4f-a1bd-203bd332ab5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:35.392005 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6165d-9437-4c4f-a1bd-203bd332ab5a None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:35.392539 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6165d-9437-4c4f-a1bd-203bd332ab5a None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:35.393023 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6165d-9437-4c4f-a1bd-203bd332ab5a None admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:35.393947 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb6165d-9437-4c4f-a1bd-203bd332ab5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:35.394454 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb6165d-9437-4c4f-a1bd-203bd332ab5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:35.397389 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb6165d-9437-4c4f-a1bd-203bd332ab5a None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:35.416631 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 297/596] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:01:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:35.417668 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb6165d-9437-4c4f-a1bd-203bd332ab5a None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:35.420832 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff52811-975a-4a1d-b27e-6047086bd9ec None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:35.458566 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1ef4a3-f644-477d-9276-28c8a7ba5879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:35.459193 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ef4a3-f644-477d-9276-28c8a7ba5879 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:35.459511 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ef4a3-f644-477d-9276-28c8a7ba5879 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:35.459786 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1ef4a3-f644-477d-9276-28c8a7ba5879 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:35.461620 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:35.470714 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1ef4a3-f644-477d-9276-28c8a7ba5879 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:35.472196 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1ef4a3-f644-477d-9276-28c8a7ba5879 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:35.487693 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-cd1ef4a3-f644-477d-9276-28c8a7ba5879 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:35.487693 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:35.487693 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:35.487693 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:35.487693 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:35.487693 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:35.487693 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:35.487693 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:35.487693 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:35.487693 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:35.487693 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:35.487693 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:35.487693 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:01:35.487693 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:01:35.487693 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:01:35.487693 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:01:35.488671 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:35.488671 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:35.488671 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:35.488671 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:35.488671 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:35.488671 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:35.488671 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:35.488671 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:35.488671 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:35.488671 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:35.488671 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:35.488671 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:35.488671 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:35.488671 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:35.488671 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:35.488671 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:35.488671 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:01:35.490003 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:01:35.490003 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:35.490003 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:35.490003 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 300/597] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:01:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:35.490003 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1ef4a3-f644-477d-9276-28c8a7ba5879 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:35.536655 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb6165d-9437-4c4f-a1bd-203bd332ab5a None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:35.557746 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:35.574975 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343b5e04-6a25-4bc9-a3b1-5ee691cac55b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:35.576773 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343b5e04-6a25-4bc9-a3b1-5ee691cac55b None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:35.577060 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343b5e04-6a25-4bc9-a3b1-5ee691cac55b None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:35.577338 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343b5e04-6a25-4bc9-a3b1-5ee691cac55b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:35.584928 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343b5e04-6a25-4bc9-a3b1-5ee691cac55b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:35.586496 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343b5e04-6a25-4bc9-a3b1-5ee691cac55b None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:35.599638 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-343b5e04-6a25-4bc9-a3b1-5ee691cac55b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:01:35.599638 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:35.599638 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:35.599638 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:35.599638 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:35.599638 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:35.599638 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:35.599638 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:35.599638 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:35.599638 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:35.599638 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:35.599638 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:35.599638 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 12:01:35.599638 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 12:01:35.599638 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 12:01:35.601812 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 12:01:35.601812 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:35.601812 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:35.601812 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:35.601812 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:35.601812 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:35.601812 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:35.601812 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:35.601812 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:35.601812 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:35.601812 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:35.601812 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:35.601812 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:35.601812 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:35.601812 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:35.601812 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:35.601812 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:35.603943 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 12:01:35.603943 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 12:01:35.603943 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 12:01:35.603943 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 12:01:35.603943 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 12:01:35.603943 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 12:01:35.603943 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:01:35.603943 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:35.603943 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 298/598] 10.222.0.61 () {56 vars in 1111 bytes} [Fri Aug 26 12:01:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:35.603943 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343b5e04-6a25-4bc9-a3b1-5ee691cac55b None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:35.605153 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1ef4a3-f644-477d-9276-28c8a7ba5879 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:35.640183 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97759e05-caf2-4274-a7d8-cf04c34d0892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:35.640811 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97759e05-caf2-4274-a7d8-cf04c34d0892 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:35.641092 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97759e05-caf2-4274-a7d8-cf04c34d0892 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:35.641366 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97759e05-caf2-4274-a7d8-cf04c34d0892 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:35.648610 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97759e05-caf2-4274-a7d8-cf04c34d0892 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:35.650125 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97759e05-caf2-4274-a7d8-cf04c34d0892 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:35.662549 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-97759e05-caf2-4274-a7d8-cf04c34d0892 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:01:35.662549 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:35.662549 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:35.662549 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:35.662549 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:35.662549 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:35.662549 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:35.662549 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:35.662549 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:35.662549 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:35.662549 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:35.662549 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:35.662549 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 12:01:35.662549 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 12:01:35.662549 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 12:01:35.664920 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 12:01:35.664920 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:35.664920 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:35.664920 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:35.664920 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:35.664920 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:35.664920 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:35.664920 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:35.664920 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:35.664920 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:35.664920 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:35.664920 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:35.664920 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:35.664920 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:35.664920 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:35.664920 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:35.664920 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:35.666870 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 12:01:35.666870 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 12:01:35.666870 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 12:01:35.666870 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 12:01:35.666870 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 12:01:35.666870 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 12:01:35.666870 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:01:35.666870 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:35.666870 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 301/599] 10.222.0.61 () {56 vars in 1111 bytes} [Fri Aug 26 12:01:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:35.666870 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97759e05-caf2-4274-a7d8-cf04c34d0892 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:35.716689 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:35.723341 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343b5e04-6a25-4bc9-a3b1-5ee691cac55b None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:35.760549 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5e1658-b11a-4a06-9a62-1bf190c89f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:35.761203 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5e1658-b11a-4a06-9a62-1bf190c89f71 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:35.761603 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5e1658-b11a-4a06-9a62-1bf190c89f71 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:35.761976 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5e1658-b11a-4a06-9a62-1bf190c89f71 None admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:35.763006 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5e1658-b11a-4a06-9a62-1bf190c89f71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:35.763190 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5e1658-b11a-4a06-9a62-1bf190c89f71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:35.764979 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5e1658-b11a-4a06-9a62-1bf190c89f71 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:35.778288 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97759e05-caf2-4274-a7d8-cf04c34d0892 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:35.782256 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 299/600] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:01:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:35.783267 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5e1658-b11a-4a06-9a62-1bf190c89f71 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:35.803664 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:35.821139 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a9da89-84c9-4d85-b182-9c78bc6eed4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:35.821752 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a9da89-84c9-4d85-b182-9c78bc6eed4c None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:35.822034 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a9da89-84c9-4d85-b182-9c78bc6eed4c None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:35.822307 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a9da89-84c9-4d85-b182-9c78bc6eed4c None admin] PATH_INFO: `/v3/projects` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:35.823176 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a9da89-84c9-4d85-b182-9c78bc6eed4c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:35.823512 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a9da89-84c9-4d85-b182-9c78bc6eed4c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:35.825805 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a9da89-84c9-4d85-b182-9c78bc6eed4c None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:35.845786 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 302/601] 10.222.0.61 () {56 vars in 1148 bytes} [Fri Aug 26 12:01:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:35.846674 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a9da89-84c9-4d85-b182-9c78bc6eed4c None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:35.911690 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5e1658-b11a-4a06-9a62-1bf190c89f71 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:35.956977 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da4578-d82d-4362-a5a3-72180c6e92c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:35.957911 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da4578-d82d-4362-a5a3-72180c6e92c8 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:35.958192 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da4578-d82d-4362-a5a3-72180c6e92c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:35.958515 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da4578-d82d-4362-a5a3-72180c6e92c8 None admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:35.959394 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57da4578-d82d-4362-a5a3-72180c6e92c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:35.959700 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57da4578-d82d-4362-a5a3-72180c6e92c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:35.961813 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57da4578-d82d-4362-a5a3-72180c6e92c8 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:35.976937 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 300/602] 10.222.0.61 () {56 vars in 1148 bytes} [Fri Aug 26 12:01:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:35.977831 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da4578-d82d-4362-a5a3-72180c6e92c8 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:35.984379 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a9da89-84c9-4d85-b182-9c78bc6eed4c None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:35.999958 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:36.021175 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52418833-a4d9-4dcc-91df-6e6112903cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:36.021800 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52418833-a4d9-4dcc-91df-6e6112903cee None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:36.022079 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52418833-a4d9-4dcc-91df-6e6112903cee None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:36.022350 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52418833-a4d9-4dcc-91df-6e6112903cee None admin] PATH_INFO: `/v3/projects/service` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:36.030067 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52418833-a4d9-4dcc-91df-6e6112903cee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:36.031600 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52418833-a4d9-4dcc-91df-6e6112903cee None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:36.043247 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-52418833-a4d9-4dcc-91df-6e6112903cee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:01:36.043247 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:36.043247 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:36.043247 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:36.043247 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:36.043247 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:36.043247 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:36.043247 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:36.043247 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:36.043247 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:36.043247 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:36.043247 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:36.043247 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 12:01:36.043247 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 12:01:36.043247 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 12:01:36.044949 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 12:01:36.044949 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:36.044949 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:36.044949 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:36.044949 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:36.044949 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:36.044949 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:36.044949 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:36.044949 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:36.044949 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:36.044949 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:36.044949 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:36.044949 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:36.044949 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:36.044949 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:36.044949 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:36.044949 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:36.045929 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 12:01:36.045929 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 12:01:36.045929 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 12:01:36.045929 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 12:01:36.045929 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 12:01:36.045929 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 12:01:36.045929 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:01:36.045929 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:36.050057 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 303/603] 10.222.0.61 () {56 vars in 1111 bytes} [Fri Aug 26 12:01:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:36.050946 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52418833-a4d9-4dcc-91df-6e6112903cee None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:36.092865 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:36.100207 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da4578-d82d-4362-a5a3-72180c6e92c8 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:36.140019 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf30428-7d26-4a8e-a78c-6da61305d215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:36.140932 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf30428-7d26-4a8e-a78c-6da61305d215 None admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:36.141413 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf30428-7d26-4a8e-a78c-6da61305d215 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:36.141927 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf30428-7d26-4a8e-a78c-6da61305d215 None admin] PATH_INFO: `/v3/projects/ecd68423a83a427681d44dbf266fce6e/users/b0972dbd5cff4610b239326425e6351c/roles/519347d2bd2c4dc1aadb0b9f8c3c8e3d` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:36.158393 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf30428-7d26-4a8e-a78c-6da61305d215 None admin] RBAC: Authorizing `identity:create_grant(project_id=ecd68423a83a427681d44dbf266fce6e, user_id=b0972dbd5cff4610b239326425e6351c, role_id=519347d2bd2c4dc1aadb0b9f8c3c8e3d)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:36.158571 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52418833-a4d9-4dcc-91df-6e6112903cee None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:36.159876 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf30428-7d26-4a8e-a78c-6da61305d215 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:36.172904 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 301/604] 10.222.0.61 () {58 vars in 1439 bytes} [Fri Aug 26 12:01:35 2022] PUT /identity/v3/projects/ecd68423a83a427681d44dbf266fce6e/users/b0972dbd5cff4610b239326425e6351c/roles/519347d2bd2c4dc1aadb0b9f8c3c8e3d => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:01:36.173966 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf30428-7d26-4a8e-a78c-6da61305d215 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:36.201202 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec272146-4540-4cae-a29b-b827a7568bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:36.202501 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec272146-4540-4cae-a29b-b827a7568bdc None admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:36.203048 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec272146-4540-4cae-a29b-b827a7568bdc None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:36.203553 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec272146-4540-4cae-a29b-b827a7568bdc None admin] PATH_INFO: `/v3/projects/ecd68423a83a427681d44dbf266fce6e/users/51239ac314e245e8a5706e65b133db3b/roles/519347d2bd2c4dc1aadb0b9f8c3c8e3d` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:36.222411 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec272146-4540-4cae-a29b-b827a7568bdc None admin] RBAC: Authorizing `identity:create_grant(project_id=ecd68423a83a427681d44dbf266fce6e, user_id=51239ac314e245e8a5706e65b133db3b, role_id=519347d2bd2c4dc1aadb0b9f8c3c8e3d)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:36.224449 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec272146-4540-4cae-a29b-b827a7568bdc None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:36.247928 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 304/605] 10.222.0.61 () {58 vars in 1439 bytes} [Fri Aug 26 12:01:36 2022] PUT /identity/v3/projects/ecd68423a83a427681d44dbf266fce6e/users/51239ac314e245e8a5706e65b133db3b/roles/519347d2bd2c4dc1aadb0b9f8c3c8e3d => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:01:36.250565 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8e4870-4d83-4688-8e7a-d68f0b8ff7a5 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:36.250854 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8e4870-4d83-4688-8e7a-d68f0b8ff7a5 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:36.251194 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8e4870-4d83-4688-8e7a-d68f0b8ff7a5 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:36.253613 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:36.254621 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 305/606] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:01:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:01:36.261627 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd8a00-20c4-4ac1-a621-63397392f087 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:36.261977 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd8a00-20c4-4ac1-a621-63397392f087 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:36.262175 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd8a00-20c4-4ac1-a621-63397392f087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:36.289309 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-00dd8a00-20c4-4ac1-a621-63397392f087 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:01:36.290612 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-00dd8a00-20c4-4ac1-a621-63397392f087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:01:36.353978 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf30428-7d26-4a8e-a78c-6da61305d215 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:36.389242 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 306/607] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:01:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:01:36.410913 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6750a1a0-e94a-4da6-9302-0e5d8e9322ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:36.411578 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6750a1a0-e94a-4da6-9302-0e5d8e9322ef None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:36.411878 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6750a1a0-e94a-4da6-9302-0e5d8e9322ef None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:36.412134 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6750a1a0-e94a-4da6-9302-0e5d8e9322ef None admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:36.412851 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6750a1a0-e94a-4da6-9302-0e5d8e9322ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:36.413160 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6750a1a0-e94a-4da6-9302-0e5d8e9322ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:36.414705 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6750a1a0-e94a-4da6-9302-0e5d8e9322ef None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:36.431242 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 302/608] 10.222.0.61 () {56 vars in 1148 bytes} [Fri Aug 26 12:01:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:36.437490 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dd8a00-20c4-4ac1-a621-63397392f087 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:36.463162 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:36.566710 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dd8a00-20c4-4ac1-a621-63397392f087 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:36.615759 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed2acda-8818-4373-8845-a2221aae4bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:36.616777 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed2acda-8818-4373-8845-a2221aae4bed None admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:36.617280 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed2acda-8818-4373-8845-a2221aae4bed None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:36.617774 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed2acda-8818-4373-8845-a2221aae4bed None admin] PATH_INFO: `/v3/projects/ecd68423a83a427681d44dbf266fce6e/users/152fd7c708114f249b9c960165f0fdfe/roles/519347d2bd2c4dc1aadb0b9f8c3c8e3d` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:36.638653 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed2acda-8818-4373-8845-a2221aae4bed None admin] RBAC: Authorizing `identity:create_grant(project_id=ecd68423a83a427681d44dbf266fce6e, user_id=152fd7c708114f249b9c960165f0fdfe, role_id=519347d2bd2c4dc1aadb0b9f8c3c8e3d)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:36.640448 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed2acda-8818-4373-8845-a2221aae4bed None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:36.675813 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 307/609] 10.222.0.61 () {58 vars in 1439 bytes} [Fri Aug 26 12:01:36 2022] PUT /identity/v3/projects/ecd68423a83a427681d44dbf266fce6e/users/152fd7c708114f249b9c960165f0fdfe/roles/519347d2bd2c4dc1aadb0b9f8c3c8e3d => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:01:36.796182 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6750a1a0-e94a-4da6-9302-0e5d8e9322ef None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:36.906501 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6750a1a0-e94a-4da6-9302-0e5d8e9322ef None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:36.931755 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:36.943365 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de6570e-e061-478a-a2cc-0336af097e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:36.944004 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de6570e-e061-478a-a2cc-0336af097e4f None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:36.944285 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de6570e-e061-478a-a2cc-0336af097e4f None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:36.944556 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de6570e-e061-478a-a2cc-0336af097e4f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:36.956833 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de6570e-e061-478a-a2cc-0336af097e4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:01:36.962014 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-2de6570e-e061-478a-a2cc-0336af097e4f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 12:01:36.962014 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:36.962014 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:36.962014 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:36.962014 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:36.962014 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:36.962014 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:36.962014 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:36.962014 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:36.962014 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:36.962014 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:36.962014 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:36.962014 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:01:36.962014 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:01:36.962014 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:01:36.962014 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:01:36.962958 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:01:36.962958 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:01:36.962958 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:01:36.962958 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:01:36.962958 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:36.962958 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:36.962958 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:36.962958 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:36.962958 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:36.962958 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:36.962958 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:36.962958 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:36.962958 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:36.962958 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:36.962958 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:36.962958 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:36.963883 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:36.963883 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:36.963883 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:36.963883 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:36.963883 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:01:36.963883 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:01:36.963883 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:01:36.963883 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:01:36.963883 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:01:36.963883 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:01:36.963883 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 12:01:36.963883 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:36.963883 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 303/610] 10.222.0.61 () {56 vars in 1102 bytes} [Fri Aug 26 12:01:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:36.970000 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed2acda-8818-4373-8845-a2221aae4bed None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:37.047511 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:37.084009 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed2acda-8818-4373-8845-a2221aae4bed None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:37.120983 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff2ba8a-be07-45fa-8bd1-cd357b86bf23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:37.121593 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff2ba8a-be07-45fa-8bd1-cd357b86bf23 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:37.121868 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff2ba8a-be07-45fa-8bd1-cd357b86bf23 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:37.122167 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff2ba8a-be07-45fa-8bd1-cd357b86bf23 None admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:37.122890 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff2ba8a-be07-45fa-8bd1-cd357b86bf23 None admin] RBAC: Adding query filter params (name=service) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:37.123192 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff2ba8a-be07-45fa-8bd1-cd357b86bf23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:37.124581 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff2ba8a-be07-45fa-8bd1-cd357b86bf23 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:37.134043 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 308/611] 10.222.0.61 () {56 vars in 1103 bytes} [Fri Aug 26 12:01:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:37.140161 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de6570e-e061-478a-a2cc-0336af097e4f None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:37.284265 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de6570e-e061-478a-a2cc-0336af097e4f None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:37.335903 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047c5f2a-6751-4e85-86de-906e3cbb8157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:37.336919 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c5f2a-6751-4e85-86de-906e3cbb8157 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:37.337432 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c5f2a-6751-4e85-86de-906e3cbb8157 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:37.337914 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c5f2a-6751-4e85-86de-906e3cbb8157 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:37.354817 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047c5f2a-6751-4e85-86de-906e3cbb8157 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:37.356693 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047c5f2a-6751-4e85-86de-906e3cbb8157 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:37.375197 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-047c5f2a-6751-4e85-86de-906e3cbb8157 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:37.375197 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:37.375197 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:37.375197 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:37.375197 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:37.375197 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:37.375197 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:37.375197 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:37.375197 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:37.375197 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:37.375197 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:37.375197 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:37.375197 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:01:37.375197 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:01:37.375197 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:01:37.375197 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:01:37.376213 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:37.376213 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:37.376213 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:37.376213 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:37.376213 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:37.376213 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:37.376213 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:37.376213 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:37.376213 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:37.376213 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:37.376213 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:37.376213 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:37.376213 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:37.376213 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:37.376213 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:37.376213 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:37.376213 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:01:37.377122 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:01:37.377122 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:37.377122 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:37.379176 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 304/612] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:01:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:37.384639 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff2ba8a-be07-45fa-8bd1-cd357b86bf23 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:37.388370 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d1acd9-f7b2-4da4-b3f9-8d7ba172e448 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:37.388740 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d1acd9-f7b2-4da4-b3f9-8d7ba172e448 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:37.389019 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d1acd9-f7b2-4da4-b3f9-8d7ba172e448 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:37.390908 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 305/613] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:01:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:01:37.396998 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc3ffe-ac05-4d32-a92e-84ed13c10212 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:37.397286 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc3ffe-ac05-4d32-a92e-84ed13c10212 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:37.397572 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc3ffe-ac05-4d32-a92e-84ed13c10212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:37.432242 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-a1fc3ffe-ac05-4d32-a92e-84ed13c10212 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:01:37.433832 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-a1fc3ffe-ac05-4d32-a92e-84ed13c10212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:01:37.502732 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:37.511821 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff2ba8a-be07-45fa-8bd1-cd357b86bf23 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:37.543686 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:37.558973 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a2e7be-f1d1-4ac4-b307-16f2cecd9e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:37.560181 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a2e7be-f1d1-4ac4-b307-16f2cecd9e6e None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:37.560607 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a2e7be-f1d1-4ac4-b307-16f2cecd9e6e None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:37.561114 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a2e7be-f1d1-4ac4-b307-16f2cecd9e6e None admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:37.561883 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a2e7be-f1d1-4ac4-b307-16f2cecd9e6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:37.562192 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a2e7be-f1d1-4ac4-b307-16f2cecd9e6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:37.571822 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a2e7be-f1d1-4ac4-b307-16f2cecd9e6e None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:37.575363 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 306/614] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:01:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 12:01:37.586921 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 309/615] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:01:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:37.594373 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fc3ffe-ac05-4d32-a92e-84ed13c10212 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:37.719628 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fc3ffe-ac05-4d32-a92e-84ed13c10212 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:37.772789 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a473df-b0b8-4563-a3fe-77bd66f21dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:37.773432 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a473df-b0b8-4563-a3fe-77bd66f21dde None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:37.773715 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a473df-b0b8-4563-a3fe-77bd66f21dde None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:37.773984 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a473df-b0b8-4563-a3fe-77bd66f21dde None admin] PATH_INFO: `/v3/users/swift` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:37.782235 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a473df-b0b8-4563-a3fe-77bd66f21dde None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:37.783709 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a473df-b0b8-4563-a3fe-77bd66f21dde None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:37.800333 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-75a473df-b0b8-4563-a3fe-77bd66f21dde None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 26 12:01:37.800333 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:37.800333 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:37.800333 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:37.800333 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:37.800333 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:37.800333 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:37.800333 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:37.800333 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:37.800333 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:37.800333 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:37.800333 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:37.800333 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 12:01:37.800333 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 12:01:37.800333 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 12:01:37.800333 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 12:01:37.801694 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:37.801694 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:37.801694 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 12:01:37.801694 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:01:37.801694 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 12:01:37.801694 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:01:37.801694 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:37.801694 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:37.801694 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:37.801694 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:37.801694 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:37.801694 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:37.801694 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:37.801694 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:37.801694 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:37.801694 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:37.801694 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:37.801694 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:37.802661 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:37.802661 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:37.802661 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 12:01:37.802661 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 12:01:37.802661 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 12:01:37.802661 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 12:01:37.802661 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 12:01:37.802661 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 12:01:37.802661 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 26 12:01:37.802661 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:37.805632 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 307/616] 10.222.0.61 () {56 vars in 1096 bytes} [Fri Aug 26 12:01:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:37.806841 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a2e7be-f1d1-4ac4-b307-16f2cecd9e6e None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:37.847196 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a473df-b0b8-4563-a3fe-77bd66f21dde None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:37.897833 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:38.014945 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a473df-b0b8-4563-a3fe-77bd66f21dde None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:38.052910 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1bd2a9-f654-4eaf-a852-6163159998b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:38.053625 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1bd2a9-f654-4eaf-a852-6163159998b2 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:38.053915 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1bd2a9-f654-4eaf-a852-6163159998b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:38.054190 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1bd2a9-f654-4eaf-a852-6163159998b2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:38.065789 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1bd2a9-f654-4eaf-a852-6163159998b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:01:38.071630 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-af1bd2a9-f654-4eaf-a852-6163159998b2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 12:01:38.071630 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:38.071630 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:38.071630 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:38.071630 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:38.071630 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:38.071630 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:38.071630 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:38.071630 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:38.071630 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:38.071630 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:38.071630 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:38.071630 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:01:38.071630 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:01:38.071630 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:01:38.071630 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:01:38.072580 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:01:38.072580 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:01:38.072580 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:01:38.072580 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:01:38.072580 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:38.072580 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:38.072580 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:38.072580 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:38.072580 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:38.072580 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:38.072580 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:38.072580 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:38.072580 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:38.072580 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:38.072580 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:38.072580 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:38.073441 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:38.073441 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:38.073441 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:38.073441 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:38.073441 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:01:38.073441 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:01:38.073441 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:01:38.073441 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:01:38.073441 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:01:38.073441 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:01:38.073441 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 12:01:38.073441 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:38.073441 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 308/617] 10.222.0.61 () {56 vars in 1102 bytes} [Fri Aug 26 12:01:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 12:01:38.078712 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a2e7be-f1d1-4ac4-b307-16f2cecd9e6e None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:38.080156 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1bd2a9-f654-4eaf-a852-6163159998b2 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:38.143571 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f571b6da-4ec2-4850-b396-129dff138b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:38.144190 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f571b6da-4ec2-4850-b396-129dff138b25 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:38.144473 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f571b6da-4ec2-4850-b396-129dff138b25 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:38.144748 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f571b6da-4ec2-4850-b396-129dff138b25 None admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:38.149048 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f571b6da-4ec2-4850-b396-129dff138b25 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:38.149365 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f571b6da-4ec2-4850-b396-129dff138b25 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:38.150832 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f571b6da-4ec2-4850-b396-129dff138b25 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:38.191261 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 310/618] 10.222.0.61 () {56 vars in 1135 bytes} [Fri Aug 26 12:01:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 385 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:38.196505 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f571b6da-4ec2-4850-b396-129dff138b25 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:38.239671 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:38.259712 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1bd2a9-f654-4eaf-a852-6163159998b2 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:38.262097 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:38.315265 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f571b6da-4ec2-4850-b396-129dff138b25 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:38.333031 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2be82b-e0ce-49aa-bd51-252819b157b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:38.333651 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2be82b-e0ce-49aa-bd51-252819b157b7 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:38.333936 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2be82b-e0ce-49aa-bd51-252819b157b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:38.334213 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2be82b-e0ce-49aa-bd51-252819b157b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:38.334990 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2be82b-e0ce-49aa-bd51-252819b157b7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:38.343232 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2be82b-e0ce-49aa-bd51-252819b157b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:38.343232 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2be82b-e0ce-49aa-bd51-252819b157b7 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:38.346759 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 309/619] 10.222.0.61 () {56 vars in 1103 bytes} [Fri Aug 26 12:01:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:38.351805 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2be82b-e0ce-49aa-bd51-252819b157b7 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:38.362580 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e316ee6-4052-46b2-8a6e-809417470d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:38.363529 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e316ee6-4052-46b2-8a6e-809417470d09 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:38.364029 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e316ee6-4052-46b2-8a6e-809417470d09 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:38.364499 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e316ee6-4052-46b2-8a6e-809417470d09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:38.375416 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e316ee6-4052-46b2-8a6e-809417470d09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:38.377299 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e316ee6-4052-46b2-8a6e-809417470d09 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:38.392870 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-3e316ee6-4052-46b2-8a6e-809417470d09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:38.392870 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:38.392870 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:38.392870 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:38.392870 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:38.392870 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:38.392870 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:38.392870 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:38.392870 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:38.392870 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:38.392870 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:38.392870 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:38.392870 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:01:38.392870 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:01:38.392870 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:01:38.392870 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:01:38.393887 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:38.393887 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:38.393887 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:38.393887 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:38.393887 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:38.393887 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:38.393887 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:38.393887 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:38.393887 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:38.393887 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:38.393887 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:38.393887 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:38.393887 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:38.393887 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:38.393887 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:38.393887 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:38.393887 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:01:38.394774 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:01:38.394774 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:38.394774 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:38.396840 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 311/620] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:01:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 12:01:38.402460 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e316ee6-4052-46b2-8a6e-809417470d09 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:38.506757 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e316ee6-4052-46b2-8a6e-809417470d09 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:38.509034 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:38.544093 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa87db4-343b-412b-97a9-0d02c42d6234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:38.545491 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa87db4-343b-412b-97a9-0d02c42d6234 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:38.545999 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa87db4-343b-412b-97a9-0d02c42d6234 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:38.546469 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa87db4-343b-412b-97a9-0d02c42d6234 None admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:38.547344 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa87db4-343b-412b-97a9-0d02c42d6234 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:38.547865 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa87db4-343b-412b-97a9-0d02c42d6234 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:38.549423 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa87db4-343b-412b-97a9-0d02c42d6234 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:38.563599 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 312/621] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:01:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:01:38.569574 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa87db4-343b-412b-97a9-0d02c42d6234 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:38.660723 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2be82b-e0ce-49aa-bd51-252819b157b7 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:38.681195 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa87db4-343b-412b-97a9-0d02c42d6234 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:38.695881 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf604c-bfb1-480d-8673-9a16dab61d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:38.696879 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf604c-bfb1-480d-8673-9a16dab61d69 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:38.697167 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf604c-bfb1-480d-8673-9a16dab61d69 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:38.697468 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf604c-bfb1-480d-8673-9a16dab61d69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:38.708900 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abf604c-bfb1-480d-8673-9a16dab61d69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:38.710357 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abf604c-bfb1-480d-8673-9a16dab61d69 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:38.718431 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:38.719664 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfd4191-2a5b-4c77-8a1a-65ac7b17d0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:38.720621 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd4191-2a5b-4c77-8a1a-65ac7b17d0f5 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:38.720895 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd4191-2a5b-4c77-8a1a-65ac7b17d0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:38.721170 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd4191-2a5b-4c77-8a1a-65ac7b17d0f5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:38.725173 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-2abf604c-bfb1-480d-8673-9a16dab61d69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:38.725173 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:38.725173 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:38.725173 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:38.725173 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:38.725173 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:38.725173 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:38.725173 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:38.725173 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:38.725173 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:38.725173 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:38.725173 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:38.725173 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:01:38.725173 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:01:38.725173 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:01:38.725173 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:01:38.726127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:38.726127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:38.726127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:38.726127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:38.726127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:38.726127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:38.726127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:38.726127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:38.726127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:38.726127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:38.726127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:38.726127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:38.726127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:38.726127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:38.726127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:38.726127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:38.726127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:01:38.727014 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:01:38.727014 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:38.727014 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:38.728218 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfd4191-2a5b-4c77-8a1a-65ac7b17d0f5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:38.729654 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfd4191-2a5b-4c77-8a1a-65ac7b17d0f5 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:38.731336 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 310/622] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:01:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:38.733353 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54857e69-46f3-4c6a-adc4-621dea2ffaea None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:38.733841 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54857e69-46f3-4c6a-adc4-621dea2ffaea None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:38.734452 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54857e69-46f3-4c6a-adc4-621dea2ffaea None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:38.735889 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 311/623] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:01:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:01:38.736811 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b72f38c-af19-46f2-89a4-907c5e4c43da None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:38.737184 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b72f38c-af19-46f2-89a4-907c5e4c43da None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:38.737455 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b72f38c-af19-46f2-89a4-907c5e4c43da None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:38.738724 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 312/624] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:01:38 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:01:38.748185 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b72f38c-af19-46f2-89a4-907c5e4c43da None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:38.749037 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-2cfd4191-2a5b-4c77-8a1a-65ac7b17d0f5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:01:38.749037 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:38.749037 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:38.749037 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:38.749037 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:38.749037 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:38.749037 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:38.749037 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:38.749037 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:38.749037 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:38.749037 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:38.749037 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:38.749037 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 12:01:38.749037 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 12:01:38.749037 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 12:01:38.749900 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 12:01:38.749900 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:38.749900 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:38.749900 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:38.749900 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:38.749900 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:38.749900 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:38.749900 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:38.749900 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:38.749900 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:38.749900 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:38.749900 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:38.749900 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:38.749900 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:38.749900 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:38.749900 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:38.749900 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:38.750797 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 12:01:38.750797 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 12:01:38.750797 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 12:01:38.750797 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 12:01:38.750797 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 12:01:38.750797 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 12:01:38.750797 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:01:38.750797 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:38.751699 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 313/625] 10.222.0.61 () {56 vars in 1111 bytes} [Fri Aug 26 12:01:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 12:01:38.752561 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f47368-f2cb-4099-8e71-a1acbd7a7e97 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:38.752842 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f47368-f2cb-4099-8e71-a1acbd7a7e97 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:38.753112 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f47368-f2cb-4099-8e71-a1acbd7a7e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:38.770902 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-53f47368-f2cb-4099-8e71-a1acbd7a7e97 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:01:38.772105 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-53f47368-f2cb-4099-8e71-a1acbd7a7e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:01:38.787458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:38.845761 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 314/626] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:01:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:01:38.847003 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f988009b-3521-4b49-8d28-c474e4eaba82 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:38.847390 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f988009b-3521-4b49-8d28-c474e4eaba82 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:38.847735 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f988009b-3521-4b49-8d28-c474e4eaba82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:38.873024 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-f988009b-3521-4b49-8d28-c474e4eaba82 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:01:38.874303 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-f988009b-3521-4b49-8d28-c474e4eaba82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:01:38.875623 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b72f38c-af19-46f2-89a4-907c5e4c43da None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:38.914280 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c976ddf3-23f3-48c7-9e21-43dcaeef25e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:38.914982 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c976ddf3-23f3-48c7-9e21-43dcaeef25e3 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:38.915355 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c976ddf3-23f3-48c7-9e21-43dcaeef25e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:38.915693 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c976ddf3-23f3-48c7-9e21-43dcaeef25e3 None admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:38.916512 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c976ddf3-23f3-48c7-9e21-43dcaeef25e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:38.916880 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c976ddf3-23f3-48c7-9e21-43dcaeef25e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:38.918374 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c976ddf3-23f3-48c7-9e21-43dcaeef25e3 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:38.936178 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 313/627] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:01:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:38.937165 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c976ddf3-23f3-48c7-9e21-43dcaeef25e3 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:38.961912 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 315/628] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:01:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:01:38.963121 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f988009b-3521-4b49-8d28-c474e4eaba82 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:39.057490 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:39.074605 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:39.111255 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f988009b-3521-4b49-8d28-c474e4eaba82 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:39.132207 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c976ddf3-23f3-48c7-9e21-43dcaeef25e3 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:39.178313 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee42a45c-96c9-4abe-a5cc-5d9ae79ffc83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:39.179045 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee42a45c-96c9-4abe-a5cc-5d9ae79ffc83 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:39.179415 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee42a45c-96c9-4abe-a5cc-5d9ae79ffc83 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:39.179750 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee42a45c-96c9-4abe-a5cc-5d9ae79ffc83 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:39.190457 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee42a45c-96c9-4abe-a5cc-5d9ae79ffc83 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:39.192302 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1001e48-2b84-4408-9fa7-33dc04032b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:39.192600 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee42a45c-96c9-4abe-a5cc-5d9ae79ffc83 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:39.193126 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1001e48-2b84-4408-9fa7-33dc04032b49 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:39.193490 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1001e48-2b84-4408-9fa7-33dc04032b49 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:39.193855 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1001e48-2b84-4408-9fa7-33dc04032b49 None admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:39.194652 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1001e48-2b84-4408-9fa7-33dc04032b49 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:39.195184 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1001e48-2b84-4408-9fa7-33dc04032b49 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:39.196832 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1001e48-2b84-4408-9fa7-33dc04032b49 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:39.206768 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-ee42a45c-96c9-4abe-a5cc-5d9ae79ffc83 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 12:01:39.206768 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:39.206768 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:39.206768 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:39.206768 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:39.206768 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:39.206768 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:39.206768 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:39.206768 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:39.206768 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:39.206768 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:39.206768 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:39.206768 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 12:01:39.206768 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 12:01:39.206768 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 12:01:39.206768 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 12:01:39.207933 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:39.207933 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:39.207933 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 12:01:39.207933 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:01:39.207933 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 12:01:39.207933 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:01:39.207933 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:39.207933 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:39.207933 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:39.207933 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:39.207933 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:39.207933 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:39.207933 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:39.207933 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:39.207933 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:39.207933 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:39.207933 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:39.207933 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:39.209178 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:39.209178 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:39.209178 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 12:01:39.209178 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 12:01:39.209178 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 12:01:39.209178 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 12:01:39.209178 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 12:01:39.209178 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 12:01:39.209178 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 12:01:39.209178 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:39.211392 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 316/629] 10.222.0.61 () {56 vars in 1093 bytes} [Fri Aug 26 12:01:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:39.212363 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 314/630] 10.222.0.61 () {56 vars in 1148 bytes} [Fri Aug 26 12:01:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:39.212604 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee42a45c-96c9-4abe-a5cc-5d9ae79ffc83 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:39.213731 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be96551-aab0-4a72-be56-dcbabc54615a None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:39.218872 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be96551-aab0-4a72-be56-dcbabc54615a None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:39.219287 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be96551-aab0-4a72-be56-dcbabc54615a None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:39.221832 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 315/631] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:01:39 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:01:39.221832 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be96551-aab0-4a72-be56-dcbabc54615a None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:39.318272 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee42a45c-96c9-4abe-a5cc-5d9ae79ffc83 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:39.332283 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be96551-aab0-4a72-be56-dcbabc54615a None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:39.350922 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e039f7a-aa17-46c3-bfba-1668da49ead8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:39.351401 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:39.351573 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e039f7a-aa17-46c3-bfba-1668da49ead8 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:39.352200 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e039f7a-aa17-46c3-bfba-1668da49ead8 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:39.352486 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e039f7a-aa17-46c3-bfba-1668da49ead8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:39.360527 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:39.363881 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e039f7a-aa17-46c3-bfba-1668da49ead8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:01:39.367112 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2316f9-81ba-49f8-af80-6cfe069b6397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:39.367746 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2316f9-81ba-49f8-af80-6cfe069b6397 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:39.368029 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2316f9-81ba-49f8-af80-6cfe069b6397 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:39.368305 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2316f9-81ba-49f8-af80-6cfe069b6397 None admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:39.369025 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2316f9-81ba-49f8-af80-6cfe069b6397 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:39.369138 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-9e039f7a-aa17-46c3-bfba-1668da49ead8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 12:01:39.369138 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:39.369138 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:39.369138 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:39.369138 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:39.369138 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:39.369138 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:39.369138 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:39.369138 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:39.369138 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:39.369138 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:39.369138 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:39.369138 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:01:39.369138 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:01:39.369138 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:01:39.369138 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:01:39.370251 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:01:39.370251 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:01:39.370251 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:01:39.370251 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:01:39.370251 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:39.370251 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:39.370251 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:39.370251 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:39.370251 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:39.370251 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:39.370251 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:39.370251 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:39.370251 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:39.370251 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:39.370251 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:39.370251 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:39.371614 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:39.371614 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:39.371614 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:39.371614 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:39.371614 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:01:39.371614 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:01:39.371614 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:01:39.371614 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:01:39.371614 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:01:39.371614 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:01:39.371614 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 12:01:39.371614 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:39.372899 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2316f9-81ba-49f8-af80-6cfe069b6397 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:39.372977 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 317/632] 10.222.0.61 () {56 vars in 1102 bytes} [Fri Aug 26 12:01:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:39.373050 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2316f9-81ba-49f8-af80-6cfe069b6397 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:39.373123 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e039f7a-aa17-46c3-bfba-1668da49ead8 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:39.396081 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 316/633] 10.222.0.61 () {56 vars in 1133 bytes} [Fri Aug 26 12:01:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:39.396081 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f32dd2-3934-45b1-b40d-5706f187fa9b None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:39.396081 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f32dd2-3934-45b1-b40d-5706f187fa9b None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:39.396081 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f32dd2-3934-45b1-b40d-5706f187fa9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:39.410414 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-35f32dd2-3934-45b1-b40d-5706f187fa9b None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:01:39.411634 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-35f32dd2-3934-45b1-b40d-5706f187fa9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:01:39.474750 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e039f7a-aa17-46c3-bfba-1668da49ead8 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:39.477624 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 317/634] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:01:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:01:39.478439 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f32dd2-3934-45b1-b40d-5706f187fa9b None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:39.509676 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9df5aeb-cb34-4831-a048-6e34a9e7b237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:39.510333 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df5aeb-cb34-4831-a048-6e34a9e7b237 None admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:39.510612 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df5aeb-cb34-4831-a048-6e34a9e7b237 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:39.510887 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df5aeb-cb34-4831-a048-6e34a9e7b237 None admin] PATH_INFO: `/v3/projects/ecd68423a83a427681d44dbf266fce6e/users/ead95a244d11432d9fe9f718e1a75f38/roles/519347d2bd2c4dc1aadb0b9f8c3c8e3d` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:39.527435 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9df5aeb-cb34-4831-a048-6e34a9e7b237 None admin] RBAC: Authorizing `identity:create_grant(project_id=ecd68423a83a427681d44dbf266fce6e, user_id=ead95a244d11432d9fe9f718e1a75f38, role_id=519347d2bd2c4dc1aadb0b9f8c3c8e3d)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:39.529700 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9df5aeb-cb34-4831-a048-6e34a9e7b237 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:39.543222 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 318/635] 10.222.0.61 () {58 vars in 1439 bytes} [Fri Aug 26 12:01:39 2022] PUT /identity/v3/projects/ecd68423a83a427681d44dbf266fce6e/users/ead95a244d11432d9fe9f718e1a75f38/roles/519347d2bd2c4dc1aadb0b9f8c3c8e3d => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:01:39.544710 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9df5aeb-cb34-4831-a048-6e34a9e7b237 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:39.601496 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f32dd2-3934-45b1-b40d-5706f187fa9b None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:39.614840 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:39.626566 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:39.648706 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb861f42-c5b1-4cb6-b00c-d2637e1f8664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:39.649694 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb861f42-c5b1-4cb6-b00c-d2637e1f8664 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:39.650167 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb861f42-c5b1-4cb6-b00c-d2637e1f8664 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:39.650621 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb861f42-c5b1-4cb6-b00c-d2637e1f8664 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:39.662395 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb861f42-c5b1-4cb6-b00c-d2637e1f8664 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:01:39.667602 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-eb861f42-c5b1-4cb6-b00c-d2637e1f8664 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 12:01:39.667602 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:39.667602 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:39.667602 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:39.667602 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:39.667602 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:39.667602 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:39.667602 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:39.667602 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:39.667602 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:39.667602 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:39.667602 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:39.667602 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:01:39.667602 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:01:39.667602 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:01:39.667602 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:01:39.668595 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:01:39.668595 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:01:39.668595 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:01:39.668595 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:01:39.668595 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:39.668595 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:39.668595 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:39.668595 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:39.668595 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:39.668595 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:39.668595 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:39.668595 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:39.668595 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:39.668595 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:39.668595 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:39.668595 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:39.670113 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:39.670113 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:39.670113 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:39.670113 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:39.670113 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:01:39.670113 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:01:39.670113 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:01:39.670113 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:01:39.670113 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:01:39.670113 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:01:39.670113 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 12:01:39.670113 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:39.672188 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 318/636] 10.222.0.61 () {56 vars in 1102 bytes} [Fri Aug 26 12:01:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:39.673162 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb861f42-c5b1-4cb6-b00c-d2637e1f8664 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:39.702061 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9df5aeb-cb34-4831-a048-6e34a9e7b237 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:39.750774 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d01f80-ce5e-4d65-96ae-987011c24296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:39.751418 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d01f80-ce5e-4d65-96ae-987011c24296 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:39.751697 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d01f80-ce5e-4d65-96ae-987011c24296 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:39.751972 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d01f80-ce5e-4d65-96ae-987011c24296 None admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:39.752877 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d01f80-ce5e-4d65-96ae-987011c24296 None admin] RBAC: Adding query filter params (name=service) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:39.753001 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d01f80-ce5e-4d65-96ae-987011c24296 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:39.754301 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d01f80-ce5e-4d65-96ae-987011c24296 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:39.767321 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 319/637] 10.222.0.61 () {56 vars in 1103 bytes} [Fri Aug 26 12:01:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:39.767321 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d01f80-ce5e-4d65-96ae-987011c24296 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:39.797003 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb861f42-c5b1-4cb6-b00c-d2637e1f8664 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:39.839955 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30b8d30-7473-48d7-bf1b-102d20a7bd43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:39.841453 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b8d30-7473-48d7-bf1b-102d20a7bd43 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:39.841934 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b8d30-7473-48d7-bf1b-102d20a7bd43 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:39.842502 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b8d30-7473-48d7-bf1b-102d20a7bd43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:39.874291 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d01f80-ce5e-4d65-96ae-987011c24296 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:39.885405 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:39.889102 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:39.914721 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808ca255-de39-4878-9695-bb1440154f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:39.915505 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ca255-de39-4878-9695-bb1440154f75 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:39.915856 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ca255-de39-4878-9695-bb1440154f75 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:39.916284 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ca255-de39-4878-9695-bb1440154f75 None admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:39.916917 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808ca255-de39-4878-9695-bb1440154f75 None admin] RBAC: Adding query filter params (name=service) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:39.917289 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808ca255-de39-4878-9695-bb1440154f75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:39.918720 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808ca255-de39-4878-9695-bb1440154f75 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:39.923313 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30b8d30-7473-48d7-bf1b-102d20a7bd43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:39.924992 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30b8d30-7473-48d7-bf1b-102d20a7bd43 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:39.928342 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 320/638] 10.222.0.61 () {56 vars in 1103 bytes} [Fri Aug 26 12:01:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:39.928893 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808ca255-de39-4878-9695-bb1440154f75 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:40.034836 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808ca255-de39-4878-9695-bb1440154f75 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:40.044947 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 319/639] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:01:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 12:01:40.044947 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30b8d30-7473-48d7-bf1b-102d20a7bd43 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:40.078382 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9692ba-be13-4a1e-84d5-b8e3241095f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:40.079015 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9692ba-be13-4a1e-84d5-b8e3241095f6 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:40.079330 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9692ba-be13-4a1e-84d5-b8e3241095f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:40.079607 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9692ba-be13-4a1e-84d5-b8e3241095f6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:40.095610 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9692ba-be13-4a1e-84d5-b8e3241095f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:01:40.100761 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-5a9692ba-be13-4a1e-84d5-b8e3241095f6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 12:01:40.100761 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:40.100761 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:40.100761 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:40.100761 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:40.100761 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:40.100761 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:40.100761 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:40.100761 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:40.100761 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:40.100761 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:40.100761 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:40.100761 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:01:40.100761 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:01:40.100761 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:01:40.100761 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:01:40.101930 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:01:40.101930 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:01:40.101930 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:01:40.101930 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:01:40.101930 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:40.101930 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:40.101930 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:40.101930 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:40.101930 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:40.101930 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:40.101930 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:40.101930 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:40.101930 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:40.101930 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:40.101930 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:40.101930 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:40.102875 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:40.102875 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:40.102875 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:40.102875 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:40.102875 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:01:40.102875 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:01:40.102875 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:01:40.102875 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:01:40.102875 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:01:40.102875 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:01:40.102875 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 12:01:40.102875 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:40.102875 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 321/640] 10.222.0.61 () {56 vars in 1102 bytes} [Fri Aug 26 12:01:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:40.103644 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9692ba-be13-4a1e-84d5-b8e3241095f6 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:40.137426 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:40.139682 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:40.177345 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30b8d30-7473-48d7-bf1b-102d20a7bd43 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:40.209604 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9692ba-be13-4a1e-84d5-b8e3241095f6 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:40.212331 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00a58ba-a522-4179-8434-5d0e2d254366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:40.213045 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00a58ba-a522-4179-8434-5d0e2d254366 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:40.213390 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00a58ba-a522-4179-8434-5d0e2d254366 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:40.213720 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00a58ba-a522-4179-8434-5d0e2d254366 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:40.225130 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00a58ba-a522-4179-8434-5d0e2d254366 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:40.226567 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00a58ba-a522-4179-8434-5d0e2d254366 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:40.241187 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-c00a58ba-a522-4179-8434-5d0e2d254366 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:40.241187 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:40.241187 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:40.241187 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:40.241187 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:40.241187 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:40.241187 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:40.241187 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:40.241187 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:40.241187 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:40.241187 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:40.241187 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:40.241187 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:01:40.241187 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:01:40.241187 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:01:40.241187 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:01:40.242325 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:40.242325 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:40.242325 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:40.242325 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:40.242325 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:40.242325 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:40.242325 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:40.242325 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:40.242325 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:40.242325 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:40.242325 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:40.242325 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:40.242325 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:40.242325 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:40.242325 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:40.242325 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:40.242325 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:01:40.243422 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:01:40.243422 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:40.243422 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:40.247192 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 320/641] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:01:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:40.248089 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00a58ba-a522-4179-8434-5d0e2d254366 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:40.248644 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d19aa25-bd7e-423d-89ed-3ea0a3bbc615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:40.249272 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d19aa25-bd7e-423d-89ed-3ea0a3bbc615 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:40.249548 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d19aa25-bd7e-423d-89ed-3ea0a3bbc615 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:40.249817 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d19aa25-bd7e-423d-89ed-3ea0a3bbc615 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:40.260733 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d19aa25-bd7e-423d-89ed-3ea0a3bbc615 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:40.262158 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d19aa25-bd7e-423d-89ed-3ea0a3bbc615 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:40.277233 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-5d19aa25-bd7e-423d-89ed-3ea0a3bbc615 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:40.277233 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:40.277233 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:40.277233 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:40.277233 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:40.277233 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:40.277233 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:40.277233 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:40.277233 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:40.277233 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:40.277233 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:40.277233 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:40.277233 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:01:40.277233 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:01:40.277233 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:01:40.277233 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:01:40.279036 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:40.279036 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:40.279036 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:40.279036 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:40.279036 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:40.279036 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:40.279036 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:40.279036 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:40.279036 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:40.279036 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:40.279036 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:40.279036 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:40.279036 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:40.279036 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:40.279036 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:40.279036 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:40.279036 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:01:40.280438 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:01:40.280438 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:40.280438 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:40.280438 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 322/642] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:01:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:40.280438 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d19aa25-bd7e-423d-89ed-3ea0a3bbc615 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:40.358652 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00a58ba-a522-4179-8434-5d0e2d254366 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:40.374171 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:40.380722 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:40.393550 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d19aa25-bd7e-423d-89ed-3ea0a3bbc615 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:40.396941 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b04a02-c54b-49f0-b226-f5aa6db04597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:40.397759 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b04a02-c54b-49f0-b226-f5aa6db04597 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:40.398226 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b04a02-c54b-49f0-b226-f5aa6db04597 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:40.398677 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b04a02-c54b-49f0-b226-f5aa6db04597 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:40.410773 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b04a02-c54b-49f0-b226-f5aa6db04597 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:40.412520 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b04a02-c54b-49f0-b226-f5aa6db04597 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:40.427193 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cf1759-47ac-4fda-a3d7-f842d9b9230a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:40.427814 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cf1759-47ac-4fda-a3d7-f842d9b9230a None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:40.428121 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cf1759-47ac-4fda-a3d7-f842d9b9230a None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:40.428379 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cf1759-47ac-4fda-a3d7-f842d9b9230a None admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:40.429028 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cf1759-47ac-4fda-a3d7-f842d9b9230a None admin] RBAC: Adding query filter params (name=service) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:40.429325 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cf1759-47ac-4fda-a3d7-f842d9b9230a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:40.430666 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cf1759-47ac-4fda-a3d7-f842d9b9230a None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:40.432216 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-96b04a02-c54b-49f0-b226-f5aa6db04597 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:40.432216 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:40.432216 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:40.432216 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:40.432216 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:40.432216 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:40.432216 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:40.432216 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:40.432216 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:40.432216 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:40.432216 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:40.432216 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:40.432216 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:01:40.432216 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:01:40.432216 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:01:40.432216 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:01:40.433165 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:40.433165 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:40.433165 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:40.433165 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:40.433165 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:40.433165 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:40.433165 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:40.433165 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:40.433165 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:40.433165 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:40.433165 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:40.433165 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:40.433165 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:40.433165 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:40.433165 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:40.433165 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:40.433165 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:01:40.434224 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:01:40.434224 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:40.434224 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:40.435882 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 321/643] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:01:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:40.436940 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b04a02-c54b-49f0-b226-f5aa6db04597 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:40.442641 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 323/644] 10.222.0.61 () {56 vars in 1103 bytes} [Fri Aug 26 12:01:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:40.443702 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cf1759-47ac-4fda-a3d7-f842d9b9230a None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:40.547079 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b04a02-c54b-49f0-b226-f5aa6db04597 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:40.551754 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cf1759-47ac-4fda-a3d7-f842d9b9230a None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:40.584528 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b234b0c2-b908-48b4-8dd3-62452f0daeb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:40.585129 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b234b0c2-b908-48b4-8dd3-62452f0daeb9 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:40.585441 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b234b0c2-b908-48b4-8dd3-62452f0daeb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:40.585670 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b234b0c2-b908-48b4-8dd3-62452f0daeb9 None admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:40.586299 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b234b0c2-b908-48b4-8dd3-62452f0daeb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:40.586597 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b234b0c2-b908-48b4-8dd3-62452f0daeb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:40.587981 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b234b0c2-b908-48b4-8dd3-62452f0daeb9 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:40.588082 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb527333-9f9b-4188-9010-44c8c79fcb5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:40.588594 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb527333-9f9b-4188-9010-44c8c79fcb5f None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:40.588874 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb527333-9f9b-4188-9010-44c8c79fcb5f None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:40.589139 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb527333-9f9b-4188-9010-44c8c79fcb5f None admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:40.589778 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb527333-9f9b-4188-9010-44c8c79fcb5f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:40.590079 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb527333-9f9b-4188-9010-44c8c79fcb5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:40.591978 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb527333-9f9b-4188-9010-44c8c79fcb5f None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:40.601163 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 322/645] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:01:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:40.601937 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b234b0c2-b908-48b4-8dd3-62452f0daeb9 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:40.613659 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 324/646] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:01:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:40.613659 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb527333-9f9b-4188-9010-44c8c79fcb5f None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:40.625116 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:40.640129 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:40.716175 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b234b0c2-b908-48b4-8dd3-62452f0daeb9 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:40.720365 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb527333-9f9b-4188-9010-44c8c79fcb5f None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:40.755621 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303cc217-e2ea-4bfe-8846-0745d5050ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:40.756252 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303cc217-e2ea-4bfe-8846-0745d5050ba9 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:40.756528 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303cc217-e2ea-4bfe-8846-0745d5050ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:40.756796 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303cc217-e2ea-4bfe-8846-0745d5050ba9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:40.760294 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1362aac-c0ad-427d-88b8-b37ca03db23d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:40.760894 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1362aac-c0ad-427d-88b8-b37ca03db23d None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:40.761171 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1362aac-c0ad-427d-88b8-b37ca03db23d None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:40.761441 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1362aac-c0ad-427d-88b8-b37ca03db23d None admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:40.762073 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1362aac-c0ad-427d-88b8-b37ca03db23d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:40.762374 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1362aac-c0ad-427d-88b8-b37ca03db23d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:40.764863 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1362aac-c0ad-427d-88b8-b37ca03db23d None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:40.767843 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303cc217-e2ea-4bfe-8846-0745d5050ba9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:40.769263 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303cc217-e2ea-4bfe-8846-0745d5050ba9 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:40.781600 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 323/647] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:01:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:40.781600 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1362aac-c0ad-427d-88b8-b37ca03db23d None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:40.788747 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-303cc217-e2ea-4bfe-8846-0745d5050ba9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:40.788747 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:40.788747 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:40.788747 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:40.788747 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:40.788747 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:40.788747 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:40.788747 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:40.788747 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:40.788747 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:40.788747 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:40.788747 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:40.788747 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:01:40.788747 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:01:40.788747 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:01:40.788747 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:01:40.789698 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:40.789698 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:40.789698 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:40.789698 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:40.789698 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:40.789698 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:40.789698 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:40.789698 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:40.789698 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:40.789698 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:40.789698 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:40.789698 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:40.789698 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:40.789698 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:40.789698 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:40.789698 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:40.789698 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:01:40.790628 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:01:40.790628 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:40.790628 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:40.790628 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 325/648] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:01:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:40.790984 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303cc217-e2ea-4bfe-8846-0745d5050ba9 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:40.855882 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:40.884801 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:40.895705 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1362aac-c0ad-427d-88b8-b37ca03db23d None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:40.902151 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303cc217-e2ea-4bfe-8846-0745d5050ba9 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:40.930707 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3ef97c-cf96-49ed-8e00-806266330e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:40.931384 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ef97c-cf96-49ed-8e00-806266330e81 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:40.931715 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ef97c-cf96-49ed-8e00-806266330e81 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:40.931971 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ef97c-cf96-49ed-8e00-806266330e81 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:40.938379 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ca3641-bfd0-48ce-9d66-f4311b0e0310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:40.939003 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ca3641-bfd0-48ce-9d66-f4311b0e0310 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:40.939316 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ca3641-bfd0-48ce-9d66-f4311b0e0310 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:40.939594 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ca3641-bfd0-48ce-9d66-f4311b0e0310 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:40.940838 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3ef97c-cf96-49ed-8e00-806266330e81 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:40.942264 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3ef97c-cf96-49ed-8e00-806266330e81 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:40.947756 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ca3641-bfd0-48ce-9d66-f4311b0e0310 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:40.949190 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ca3641-bfd0-48ce-9d66-f4311b0e0310 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:40.953987 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-ba3ef97c-cf96-49ed-8e00-806266330e81 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 26 12:01:40.953987 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:40.953987 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:40.953987 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:40.953987 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:40.953987 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:40.953987 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:40.953987 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:40.953987 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:40.953987 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:40.953987 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:40.953987 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:40.953987 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 12:01:40.953987 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 12:01:40.953987 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 12:01:40.953987 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 12:01:40.954963 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:40.954963 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:40.954963 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 12:01:40.954963 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:01:40.954963 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 12:01:40.954963 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:01:40.954963 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:40.954963 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:40.954963 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:40.954963 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:40.954963 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:40.954963 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:40.954963 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:40.954963 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:40.954963 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:40.954963 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:40.954963 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:40.954963 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:40.956246 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:40.956246 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:40.956246 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 12:01:40.956246 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 12:01:40.956246 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 12:01:40.956246 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 12:01:40.956246 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 12:01:40.956246 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 12:01:40.956246 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 26 12:01:40.956246 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:40.956246 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 324/649] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:01:40 2022] GET /identity/v3/users/cinder => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:40.956246 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3ef97c-cf96-49ed-8e00-806266330e81 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:40.967592 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-a3ca3641-bfd0-48ce-9d66-f4311b0e0310 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 26 12:01:40.967592 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:40.967592 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:40.967592 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:40.967592 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:40.967592 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:40.967592 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:40.967592 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:40.967592 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:40.967592 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:40.967592 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:40.967592 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:40.967592 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 12:01:40.967592 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 12:01:40.967592 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 12:01:40.967592 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 12:01:40.968524 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:40.968524 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:40.968524 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 12:01:40.968524 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:01:40.968524 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 12:01:40.968524 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:01:40.968524 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:40.968524 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:40.968524 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:40.968524 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:40.968524 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:40.968524 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:40.968524 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:40.968524 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:40.968524 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:40.968524 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:40.968524 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:40.968524 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:40.969485 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:40.969485 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:40.969485 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 12:01:40.969485 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 12:01:40.969485 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 12:01:40.969485 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 12:01:40.969485 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 12:01:40.969485 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 12:01:40.969485 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 26 12:01:40.969485 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:40.976071 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 326/650] 10.222.0.61 () {56 vars in 1102 bytes} [Fri Aug 26 12:01:40 2022] GET /identity/v3/users/neutron => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:40.976071 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ca3641-bfd0-48ce-9d66-f4311b0e0310 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:41.064999 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3ef97c-cf96-49ed-8e00-806266330e81 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:41.081165 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ca3641-bfd0-48ce-9d66-f4311b0e0310 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:41.099791 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c66570-7623-44b0-ab97-2440cd7a716f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:41.100551 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c66570-7623-44b0-ab97-2440cd7a716f None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:41.100834 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c66570-7623-44b0-ab97-2440cd7a716f None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:41.101107 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c66570-7623-44b0-ab97-2440cd7a716f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:41.102908 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:41.109020 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c66570-7623-44b0-ab97-2440cd7a716f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:41.110660 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c66570-7623-44b0-ab97-2440cd7a716f None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:41.118273 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cc1feb-9624-4ee7-8c2d-523f9e105c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:41.118872 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc1feb-9624-4ee7-8c2d-523f9e105c11 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:41.119150 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc1feb-9624-4ee7-8c2d-523f9e105c11 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:41.119450 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc1feb-9624-4ee7-8c2d-523f9e105c11 None admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:41.120089 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cc1feb-9624-4ee7-8c2d-523f9e105c11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:41.120388 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cc1feb-9624-4ee7-8c2d-523f9e105c11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:41.121734 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cc1feb-9624-4ee7-8c2d-523f9e105c11 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:41.123507 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-71c66570-7623-44b0-ab97-2440cd7a716f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:01:41.123507 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:41.123507 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:41.123507 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:41.123507 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:41.123507 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:41.123507 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:41.123507 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:41.123507 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:41.123507 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:41.123507 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:41.123507 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:41.123507 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 12:01:41.123507 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 12:01:41.123507 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 12:01:41.124546 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 12:01:41.124546 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:41.124546 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:41.124546 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:41.124546 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:41.124546 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:41.124546 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:41.124546 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:41.124546 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:41.124546 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:41.124546 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:41.124546 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:41.124546 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:41.124546 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:41.124546 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:41.124546 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:41.124546 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:41.125725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 12:01:41.125725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 12:01:41.125725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 12:01:41.125725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 12:01:41.125725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 12:01:41.125725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 12:01:41.125725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:01:41.125725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:41.132094 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 325/651] 10.222.0.61 () {56 vars in 1111 bytes} [Fri Aug 26 12:01:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:41.133405 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c66570-7623-44b0-ab97-2440cd7a716f None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:41.135433 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 327/652] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:01:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:41.136250 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cc1feb-9624-4ee7-8c2d-523f9e105c11 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:41.154332 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:41.241128 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c66570-7623-44b0-ab97-2440cd7a716f None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:41.243628 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cc1feb-9624-4ee7-8c2d-523f9e105c11 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:41.275036 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0f2050-d9cf-4554-88c9-d0081b6c8031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:41.275668 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0f2050-d9cf-4554-88c9-d0081b6c8031 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:41.275946 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0f2050-d9cf-4554-88c9-d0081b6c8031 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:41.276215 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0f2050-d9cf-4554-88c9-d0081b6c8031 None admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:41.276932 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0f2050-d9cf-4554-88c9-d0081b6c8031 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:41.277234 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0f2050-d9cf-4554-88c9-d0081b6c8031 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:41.278728 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0f2050-d9cf-4554-88c9-d0081b6c8031 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:41.281789 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0382e22f-1995-4905-87dd-c383802484ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:41.282394 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0382e22f-1995-4905-87dd-c383802484ac None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:41.282675 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0382e22f-1995-4905-87dd-c383802484ac None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:41.282946 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0382e22f-1995-4905-87dd-c383802484ac None admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:41.283701 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0382e22f-1995-4905-87dd-c383802484ac None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:41.284005 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0382e22f-1995-4905-87dd-c383802484ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:41.285419 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0382e22f-1995-4905-87dd-c383802484ac None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:41.296644 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 326/653] 10.222.0.61 () {56 vars in 1137 bytes} [Fri Aug 26 12:01:41 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:41.297763 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0f2050-d9cf-4554-88c9-d0081b6c8031 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:41.304829 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 328/654] 10.222.0.61 () {56 vars in 1139 bytes} [Fri Aug 26 12:01:41 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:41.305909 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0382e22f-1995-4905-87dd-c383802484ac None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:41.348444 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:41.374339 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:41.411324 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0f2050-d9cf-4554-88c9-d0081b6c8031 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:41.411662 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0382e22f-1995-4905-87dd-c383802484ac None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:41.446764 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65cb2a4-cc19-4c0c-8896-79511b867043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:41.447637 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65cb2a4-cc19-4c0c-8896-79511b867043 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:41.447910 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65cb2a4-cc19-4c0c-8896-79511b867043 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:41.448175 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65cb2a4-cc19-4c0c-8896-79511b867043 None admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:41.448754 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d31f8c3-05bf-44f4-aae7-9adf94a6a7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:41.448869 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65cb2a4-cc19-4c0c-8896-79511b867043 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:41.449139 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65cb2a4-cc19-4c0c-8896-79511b867043 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:41.449383 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d31f8c3-05bf-44f4-aae7-9adf94a6a7e3 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:41.449667 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d31f8c3-05bf-44f4-aae7-9adf94a6a7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:41.449945 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d31f8c3-05bf-44f4-aae7-9adf94a6a7e3 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:41.451193 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65cb2a4-cc19-4c0c-8896-79511b867043 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:41.458644 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d31f8c3-05bf-44f4-aae7-9adf94a6a7e3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:41.460574 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d31f8c3-05bf-44f4-aae7-9adf94a6a7e3 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:41.467514 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 327/655] 10.222.0.61 () {56 vars in 1148 bytes} [Fri Aug 26 12:01:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:41.467514 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65cb2a4-cc19-4c0c-8896-79511b867043 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:41.477390 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-5d31f8c3-05bf-44f4-aae7-9adf94a6a7e3 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 26 12:01:41.477390 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:41.477390 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:41.477390 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:41.477390 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:41.477390 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:41.477390 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:41.477390 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:41.477390 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:41.477390 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:41.477390 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:41.477390 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:41.477390 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 12:01:41.477390 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 12:01:41.477390 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 12:01:41.477390 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 12:01:41.478363 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:41.478363 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:41.478363 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 12:01:41.478363 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:01:41.478363 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 12:01:41.478363 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:01:41.478363 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:41.478363 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:41.478363 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:41.478363 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:41.478363 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:41.478363 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:41.478363 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:41.478363 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:41.478363 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:41.478363 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:41.478363 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:41.478363 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:41.480246 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:41.480246 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:41.480246 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 12:01:41.480246 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 12:01:41.480246 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 12:01:41.480246 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 12:01:41.480246 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 12:01:41.480246 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 12:01:41.480246 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 26 12:01:41.480246 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:41.480246 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 329/656] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:01:41 2022] GET /identity/v3/users/glance => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:41.480246 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d31f8c3-05bf-44f4-aae7-9adf94a6a7e3 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:41.576015 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65cb2a4-cc19-4c0c-8896-79511b867043 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:41.579940 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:41.583717 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d31f8c3-05bf-44f4-aae7-9adf94a6a7e3 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:41.603534 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:41.610101 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c15601-d4f9-4b8f-ac41-7f54282c5cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:41.610700 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c15601-d4f9-4b8f-ac41-7f54282c5cd7 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:41.610978 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c15601-d4f9-4b8f-ac41-7f54282c5cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:41.611272 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c15601-d4f9-4b8f-ac41-7f54282c5cd7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:41.624361 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f38af-7123-4043-af33-be1db2130178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:41.624970 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f38af-7123-4043-af33-be1db2130178 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:41.625254 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f38af-7123-4043-af33-be1db2130178 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:41.625526 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f38af-7123-4043-af33-be1db2130178 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:41.684666 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c15601-d4f9-4b8f-ac41-7f54282c5cd7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:41.686118 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c15601-d4f9-4b8f-ac41-7f54282c5cd7 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:41.692604 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5f38af-7123-4043-af33-be1db2130178 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:41.694041 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5f38af-7123-4043-af33-be1db2130178 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:41.788804 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 328/657] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:01:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 12:01:41.789651 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c15601-d4f9-4b8f-ac41-7f54282c5cd7 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:41.796494 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:41.803471 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 330/658] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:01:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 12:01:41.804616 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f38af-7123-4043-af33-be1db2130178 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:41.819118 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:41.887350 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c15601-d4f9-4b8f-ac41-7f54282c5cd7 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:41.902991 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f38af-7123-4043-af33-be1db2130178 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:41.921734 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ce2ae4-4c05-4232-a614-3b042fd583b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:41.922335 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ce2ae4-4c05-4232-a614-3b042fd583b5 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:41.922611 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ce2ae4-4c05-4232-a614-3b042fd583b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:41.922907 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ce2ae4-4c05-4232-a614-3b042fd583b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:41.923652 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ce2ae4-4c05-4232-a614-3b042fd583b5 None admin] RBAC: Adding query filter params (role.id=519347d2bd2c4dc1aadb0b9f8c3c8e3d, scope.project.id=ecd68423a83a427681d44dbf266fce6e, user.id=7aacd7d6d17e4f10ba5c9107a12beafa) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:41.923953 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ce2ae4-4c05-4232-a614-3b042fd583b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:41.925479 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ce2ae4-4c05-4232-a614-3b042fd583b5 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:41.931016 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 329/659] 10.222.0.61 () {56 vars in 1374 bytes} [Fri Aug 26 12:01:41 2022] GET /identity/v3/role_assignments?user.id=7aacd7d6d17e4f10ba5c9107a12beafa&scope.project.id=ecd68423a83a427681d44dbf266fce6e&role.id=519347d2bd2c4dc1aadb0b9f8c3c8e3d => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:41.932303 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea559013-49d7-45c7-9b2c-fe68e8cec1b5 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:41.932596 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea559013-49d7-45c7-9b2c-fe68e8cec1b5 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:41.932862 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea559013-49d7-45c7-9b2c-fe68e8cec1b5 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:41.934326 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 330/660] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:01:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:01:41.935377 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea559013-49d7-45c7-9b2c-fe68e8cec1b5 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:41.946747 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc211274-04cc-4439-b656-bf62021b2dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:41.947417 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc211274-04cc-4439-b656-bf62021b2dac None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:41.947704 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc211274-04cc-4439-b656-bf62021b2dac None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:41.947972 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc211274-04cc-4439-b656-bf62021b2dac None admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:41.948757 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc211274-04cc-4439-b656-bf62021b2dac None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:41.949030 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc211274-04cc-4439-b656-bf62021b2dac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:41.951059 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc211274-04cc-4439-b656-bf62021b2dac None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:41.975390 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 331/661] 10.222.0.61 () {56 vars in 1137 bytes} [Fri Aug 26 12:01:41 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:41.975390 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc211274-04cc-4439-b656-bf62021b2dac None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:42.036986 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:42.062555 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:42.070667 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea559013-49d7-45c7-9b2c-fe68e8cec1b5 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:42.090448 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc211274-04cc-4439-b656-bf62021b2dac None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:42.113035 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51273ff0-760f-463c-93dd-debbc205c6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:42.114116 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51273ff0-760f-463c-93dd-debbc205c6fc None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:42.114404 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51273ff0-760f-463c-93dd-debbc205c6fc None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:42.114684 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51273ff0-760f-463c-93dd-debbc205c6fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:42.126483 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51273ff0-760f-463c-93dd-debbc205c6fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:42.128097 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51273ff0-760f-463c-93dd-debbc205c6fc None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:42.133737 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc89129-a350-4251-8b87-b2950ac836bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:42.134376 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc89129-a350-4251-8b87-b2950ac836bd None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:42.134661 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc89129-a350-4251-8b87-b2950ac836bd None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:42.134932 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc89129-a350-4251-8b87-b2950ac836bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:42.147122 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-51273ff0-760f-463c-93dd-debbc205c6fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:42.147122 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:42.147122 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:42.147122 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:42.147122 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:42.147122 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:42.147122 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:42.147122 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:42.147122 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:42.147122 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:42.147122 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:42.147122 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:42.147122 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:01:42.147122 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:01:42.147122 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:01:42.147122 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:01:42.148127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:42.148127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:42.148127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:42.148127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:42.148127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:42.148127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:42.148127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:42.148127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:42.148127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:42.148127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:42.148127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:42.148127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:42.148127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:42.148127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:42.148127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:42.148127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:42.148127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:01:42.149045 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:01:42.149045 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:42.149045 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:42.149213 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc89129-a350-4251-8b87-b2950ac836bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:42.149302 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 331/662] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:01:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:42.149372 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc89129-a350-4251-8b87-b2950ac836bd None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:42.149844 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98908176-8a4e-4407-a663-a32a5c5df73f None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:42.149953 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98908176-8a4e-4407-a663-a32a5c5df73f None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:42.150195 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98908176-8a4e-4407-a663-a32a5c5df73f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:42.166268 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-2cc89129-a350-4251-8b87-b2950ac836bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:42.166268 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:42.166268 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:42.166268 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:42.166268 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:42.166268 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:42.166268 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:42.166268 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:42.166268 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:42.166268 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:42.166268 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:42.166268 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:42.166268 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:01:42.166268 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:01:42.166268 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:01:42.166268 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:01:42.167881 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:42.167881 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:42.167881 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:42.167881 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:42.167881 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:42.167881 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:42.167881 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:42.167881 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:42.167881 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:42.167881 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:42.167881 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:42.167881 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:42.167881 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:42.167881 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:42.167881 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:42.167881 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:42.167881 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:01:42.168775 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:01:42.168775 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:42.168775 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:42.168775 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 332/663] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:01:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:42.168775 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc89129-a350-4251-8b87-b2950ac836bd None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:42.174630 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-98908176-8a4e-4407-a663-a32a5c5df73f None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:01:42.174630 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-98908176-8a4e-4407-a663-a32a5c5df73f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:01:42.247273 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 332/664] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:01:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:01:42.247273 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98908176-8a4e-4407-a663-a32a5c5df73f None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:42.301519 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc89129-a350-4251-8b87-b2950ac836bd None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:42.311461 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:42.333682 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:42.342489 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd72caf-b5a5-4713-bd67-38cca4cbfd69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:42.343448 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd72caf-b5a5-4713-bd67-38cca4cbfd69 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:42.343976 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd72caf-b5a5-4713-bd67-38cca4cbfd69 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:42.344459 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd72caf-b5a5-4713-bd67-38cca4cbfd69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:42.361051 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98908176-8a4e-4407-a663-a32a5c5df73f None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:42.398382 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c75a8ac-b844-42cf-b56b-e4e8cc315af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:42.399013 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c75a8ac-b844-42cf-b56b-e4e8cc315af9 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:42.399397 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c75a8ac-b844-42cf-b56b-e4e8cc315af9 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:42.399745 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c75a8ac-b844-42cf-b56b-e4e8cc315af9 None admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:42.400458 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c75a8ac-b844-42cf-b56b-e4e8cc315af9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:42.400831 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c75a8ac-b844-42cf-b56b-e4e8cc315af9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:42.402299 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c75a8ac-b844-42cf-b56b-e4e8cc315af9 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:42.419297 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 333/665] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:01:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:42.420262 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c75a8ac-b844-42cf-b56b-e4e8cc315af9 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:42.423497 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd72caf-b5a5-4713-bd67-38cca4cbfd69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:42.425276 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd72caf-b5a5-4713-bd67-38cca4cbfd69 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:42.523787 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c75a8ac-b844-42cf-b56b-e4e8cc315af9 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:42.536540 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 333/666] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:01:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 12:01:42.541773 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd72caf-b5a5-4713-bd67-38cca4cbfd69 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:42.552868 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:42.562495 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64222bc6-d9a1-44b6-94ce-b26112089f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:42.563105 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64222bc6-d9a1-44b6-94ce-b26112089f5f None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:42.563406 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64222bc6-d9a1-44b6-94ce-b26112089f5f None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:42.563680 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64222bc6-d9a1-44b6-94ce-b26112089f5f None admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:42.564434 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64222bc6-d9a1-44b6-94ce-b26112089f5f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:42.564740 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64222bc6-d9a1-44b6-94ce-b26112089f5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:42.566109 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64222bc6-d9a1-44b6-94ce-b26112089f5f None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:42.569378 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:42.586110 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 334/667] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:01:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:42.586110 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64222bc6-d9a1-44b6-94ce-b26112089f5f None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:42.648353 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd72caf-b5a5-4713-bd67-38cca4cbfd69 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:42.683345 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a44b6b7-9bea-4e0c-bac8-d5685c876f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:42.683913 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a44b6b7-9bea-4e0c-bac8-d5685c876f01 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:42.684200 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a44b6b7-9bea-4e0c-bac8-d5685c876f01 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:42.684494 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a44b6b7-9bea-4e0c-bac8-d5685c876f01 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:42.692491 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a44b6b7-9bea-4e0c-bac8-d5685c876f01 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:42.692612 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64222bc6-d9a1-44b6-94ce-b26112089f5f None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:42.693944 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a44b6b7-9bea-4e0c-bac8-d5685c876f01 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:42.705402 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-5a44b6b7-9bea-4e0c-bac8-d5685c876f01 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:01:42.705402 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:42.705402 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:42.705402 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:42.705402 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:42.705402 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:42.705402 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:42.705402 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:42.705402 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:42.705402 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:42.705402 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:42.705402 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:42.705402 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 12:01:42.705402 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 12:01:42.705402 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 12:01:42.706290 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 12:01:42.706290 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:42.706290 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:42.706290 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:42.706290 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:42.706290 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:42.706290 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:42.706290 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:42.706290 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:42.706290 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:42.706290 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:42.706290 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:42.706290 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:42.706290 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:42.706290 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:42.706290 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:42.706290 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:42.707245 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 12:01:42.707245 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 12:01:42.707245 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 12:01:42.707245 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 12:01:42.707245 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 12:01:42.707245 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 12:01:42.707245 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:01:42.707245 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:42.707245 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 334/668] 10.222.0.61 () {56 vars in 1111 bytes} [Fri Aug 26 12:01:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:42.707245 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a44b6b7-9bea-4e0c-bac8-d5685c876f01 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:42.733487 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a51e4fb-d23a-4d2b-baa4-7c9b7994d8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:42.734141 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a51e4fb-d23a-4d2b-baa4-7c9b7994d8bd None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:42.734422 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a51e4fb-d23a-4d2b-baa4-7c9b7994d8bd None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:42.734692 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a51e4fb-d23a-4d2b-baa4-7c9b7994d8bd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:42.751482 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a51e4fb-d23a-4d2b-baa4-7c9b7994d8bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:01:42.756080 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-6a51e4fb-d23a-4d2b-baa4-7c9b7994d8bd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 12:01:42.756080 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:42.756080 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:42.756080 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:42.756080 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:42.756080 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:42.756080 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:42.756080 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:42.756080 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:42.756080 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:42.756080 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:42.756080 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:42.756080 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:01:42.756080 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:01:42.756080 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:01:42.756080 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:01:42.757040 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:01:42.757040 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:01:42.757040 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:01:42.757040 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:01:42.757040 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:42.757040 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:42.757040 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:42.757040 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:42.757040 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:42.757040 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:42.757040 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:42.757040 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:42.757040 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:42.757040 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:42.757040 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:42.757040 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:42.757889 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:42.757889 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:42.757889 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:42.757889 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:42.757889 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:01:42.757889 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:01:42.757889 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:01:42.757889 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:01:42.757889 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:01:42.757889 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:01:42.757889 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 12:01:42.757889 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:42.762188 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 335/669] 10.222.0.61 () {56 vars in 1102 bytes} [Fri Aug 26 12:01:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:42.763269 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a51e4fb-d23a-4d2b-baa4-7c9b7994d8bd None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:42.782732 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:42.819714 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a44b6b7-9bea-4e0c-bac8-d5685c876f01 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:42.823717 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:42.855385 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765ce50c-9f3d-49e3-a4d4-78b0f3b503f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:42.856017 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ce50c-9f3d-49e3-a4d4-78b0f3b503f4 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:42.856300 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ce50c-9f3d-49e3-a4d4-78b0f3b503f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:42.857235 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ce50c-9f3d-49e3-a4d4-78b0f3b503f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:42.867842 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765ce50c-9f3d-49e3-a4d4-78b0f3b503f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:42.869279 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765ce50c-9f3d-49e3-a4d4-78b0f3b503f4 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:42.873738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a51e4fb-d23a-4d2b-baa4-7c9b7994d8bd None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:42.884991 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-765ce50c-9f3d-49e3-a4d4-78b0f3b503f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:42.884991 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:42.884991 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:42.884991 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:42.884991 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:42.884991 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:42.884991 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:42.884991 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:42.884991 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:42.884991 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:42.884991 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:42.884991 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:42.884991 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:01:42.884991 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:01:42.884991 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:01:42.884991 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:01:42.886131 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:42.886131 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:42.886131 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:42.886131 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:42.886131 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:42.886131 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:42.886131 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:42.886131 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:42.886131 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:42.886131 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:42.886131 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:42.886131 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:42.886131 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:42.886131 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:42.886131 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:42.886131 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:42.886131 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:01:42.887256 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:01:42.887256 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:42.887256 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:42.890788 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 335/670] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:01:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:42.891647 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765ce50c-9f3d-49e3-a4d4-78b0f3b503f4 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:42.913619 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbbba71-a9a3-4af7-805c-8336eca98c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:42.914245 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbbba71-a9a3-4af7-805c-8336eca98c5f None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:42.914528 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbbba71-a9a3-4af7-805c-8336eca98c5f None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:42.914799 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbbba71-a9a3-4af7-805c-8336eca98c5f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:42.921964 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbbba71-a9a3-4af7-805c-8336eca98c5f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:42.923424 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbbba71-a9a3-4af7-805c-8336eca98c5f None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:42.934706 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-1bbbba71-a9a3-4af7-805c-8336eca98c5f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:01:42.934706 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:42.934706 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:42.934706 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:42.934706 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:42.934706 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:42.934706 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:42.934706 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:42.934706 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:42.934706 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:42.934706 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:42.934706 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:42.934706 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 12:01:42.934706 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 12:01:42.934706 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 12:01:42.935739 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 12:01:42.935739 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:42.935739 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:42.935739 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:42.935739 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:42.935739 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:42.935739 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:42.935739 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:42.935739 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:42.935739 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:42.935739 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:42.935739 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:42.935739 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:42.935739 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:42.935739 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:42.935739 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:42.935739 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:42.936725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 12:01:42.936725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 12:01:42.936725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 12:01:42.936725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 12:01:42.936725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 12:01:42.936725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 12:01:42.936725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:01:42.936725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:42.936725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 336/671] 10.222.0.61 () {56 vars in 1111 bytes} [Fri Aug 26 12:01:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:42.936725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbbba71-a9a3-4af7-805c-8336eca98c5f None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:43.005538 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765ce50c-9f3d-49e3-a4d4-78b0f3b503f4 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:43.026425 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:43.042182 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6184eb-89e8-47dc-8198-633dd4302b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:43.042783 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6184eb-89e8-47dc-8198-633dd4302b2a None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:43.043062 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6184eb-89e8-47dc-8198-633dd4302b2a None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:43.043370 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6184eb-89e8-47dc-8198-633dd4302b2a None admin] PATH_INFO: `/v3/projects` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:43.044185 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6184eb-89e8-47dc-8198-633dd4302b2a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:43.044497 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6184eb-89e8-47dc-8198-633dd4302b2a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:43.046082 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbbba71-a9a3-4af7-805c-8336eca98c5f None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:43.046192 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6184eb-89e8-47dc-8198-633dd4302b2a None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:43.063340 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 336/672] 10.222.0.61 () {56 vars in 1148 bytes} [Fri Aug 26 12:01:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:43.063340 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6184eb-89e8-47dc-8198-633dd4302b2a None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:43.069281 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:43.092113 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07864897-8878-4faa-a87e-fa6ff49ea3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:43.092906 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07864897-8878-4faa-a87e-fa6ff49ea3f6 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:43.093179 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07864897-8878-4faa-a87e-fa6ff49ea3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:43.093446 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07864897-8878-4faa-a87e-fa6ff49ea3f6 None admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:43.094100 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07864897-8878-4faa-a87e-fa6ff49ea3f6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:43.094397 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07864897-8878-4faa-a87e-fa6ff49ea3f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:43.095787 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07864897-8878-4faa-a87e-fa6ff49ea3f6 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:43.109387 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 337/673] 10.222.0.61 () {56 vars in 1103 bytes} [Fri Aug 26 12:01:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:43.109387 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07864897-8878-4faa-a87e-fa6ff49ea3f6 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:43.181922 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6184eb-89e8-47dc-8198-633dd4302b2a None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:43.211998 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07864897-8878-4faa-a87e-fa6ff49ea3f6 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:43.219806 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bad58fe-258d-4116-aaed-9585f5b9f66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:43.220413 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bad58fe-258d-4116-aaed-9585f5b9f66f None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:43.220699 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bad58fe-258d-4116-aaed-9585f5b9f66f None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:43.220973 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bad58fe-258d-4116-aaed-9585f5b9f66f None admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:43.221974 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bad58fe-258d-4116-aaed-9585f5b9f66f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:43.222286 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bad58fe-258d-4116-aaed-9585f5b9f66f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:43.224265 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bad58fe-258d-4116-aaed-9585f5b9f66f None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:43.237594 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 337/674] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:01:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:43.238449 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bad58fe-258d-4116-aaed-9585f5b9f66f None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:43.251417 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6d5e4c-424f-4316-9b21-766e26fcd2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:43.252027 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6d5e4c-424f-4316-9b21-766e26fcd2ab None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:43.252556 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6d5e4c-424f-4316-9b21-766e26fcd2ab None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:43.252556 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6d5e4c-424f-4316-9b21-766e26fcd2ab None admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:43.253263 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6d5e4c-424f-4316-9b21-766e26fcd2ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:43.253633 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6d5e4c-424f-4316-9b21-766e26fcd2ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:43.255001 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6d5e4c-424f-4316-9b21-766e26fcd2ab None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:43.268956 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 338/675] 10.222.0.61 () {56 vars in 1148 bytes} [Fri Aug 26 12:01:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:43.269756 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6d5e4c-424f-4316-9b21-766e26fcd2ab None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:43.289164 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:43.319532 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:43.351715 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bad58fe-258d-4116-aaed-9585f5b9f66f None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:43.384881 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6d5e4c-424f-4316-9b21-766e26fcd2ab None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:43.386972 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce66031e-b9fa-4401-b534-3f49aa633530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:43.388186 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce66031e-b9fa-4401-b534-3f49aa633530 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:43.388467 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce66031e-b9fa-4401-b534-3f49aa633530 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:43.388817 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce66031e-b9fa-4401-b534-3f49aa633530 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:43.389535 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce66031e-b9fa-4401-b534-3f49aa633530 None admin] RBAC: Adding query filter params (role.id=519347d2bd2c4dc1aadb0b9f8c3c8e3d, scope.project.id=ecd68423a83a427681d44dbf266fce6e, user.id=51239ac314e245e8a5706e65b133db3b) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:43.389863 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce66031e-b9fa-4401-b534-3f49aa633530 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:43.391294 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce66031e-b9fa-4401-b534-3f49aa633530 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:43.398433 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 338/676] 10.222.0.61 () {56 vars in 1374 bytes} [Fri Aug 26 12:01:43 2022] GET /identity/v3/role_assignments?user.id=51239ac314e245e8a5706e65b133db3b&scope.project.id=ecd68423a83a427681d44dbf266fce6e&role.id=519347d2bd2c4dc1aadb0b9f8c3c8e3d => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:43.399497 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce66031e-b9fa-4401-b534-3f49aa633530 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:43.438762 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f09f8b-00ba-45b0-a8d2-758fee4eb6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:43.439879 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f09f8b-00ba-45b0-a8d2-758fee4eb6ba None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:43.440378 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f09f8b-00ba-45b0-a8d2-758fee4eb6ba None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:43.440840 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f09f8b-00ba-45b0-a8d2-758fee4eb6ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:43.453598 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f09f8b-00ba-45b0-a8d2-758fee4eb6ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:43.455660 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f09f8b-00ba-45b0-a8d2-758fee4eb6ba None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:43.473186 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-36f09f8b-00ba-45b0-a8d2-758fee4eb6ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:43.473186 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:43.473186 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:43.473186 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:43.473186 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:43.473186 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:43.473186 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:43.473186 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:43.473186 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:43.473186 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:43.473186 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:43.473186 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:43.473186 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:01:43.473186 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:01:43.473186 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:01:43.473186 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:01:43.474171 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:43.474171 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:43.474171 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:43.474171 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:43.474171 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:43.474171 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:43.474171 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:43.474171 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:43.474171 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:43.474171 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:43.474171 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:43.474171 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:43.474171 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:43.474171 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:43.474171 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:43.474171 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:43.474171 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:01:43.475069 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:01:43.475069 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:43.475069 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:43.477520 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 339/677] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:01:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:43.478678 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f09f8b-00ba-45b0-a8d2-758fee4eb6ba None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:43.529560 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce66031e-b9fa-4401-b534-3f49aa633530 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:43.533274 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:43.571089 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93530cc-cadd-44ff-b9f6-b3ca6d0bc18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:43.571793 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93530cc-cadd-44ff-b9f6-b3ca6d0bc18f None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:43.572078 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93530cc-cadd-44ff-b9f6-b3ca6d0bc18f None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:43.572358 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93530cc-cadd-44ff-b9f6-b3ca6d0bc18f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:43.581442 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93530cc-cadd-44ff-b9f6-b3ca6d0bc18f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:43.583899 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93530cc-cadd-44ff-b9f6-b3ca6d0bc18f None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:43.592958 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:43.599594 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-b93530cc-cadd-44ff-b9f6-b3ca6d0bc18f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:01:43.599594 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:43.599594 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:43.599594 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:43.599594 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:43.599594 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:43.599594 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:43.599594 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:43.599594 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:43.599594 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:43.599594 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:43.599594 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:43.599594 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 12:01:43.599594 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 12:01:43.599594 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 12:01:43.600544 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 12:01:43.600544 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:43.600544 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:43.600544 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:43.600544 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:43.600544 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:43.600544 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:43.600544 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:43.600544 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:43.600544 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:43.600544 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:43.600544 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:43.600544 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:43.600544 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:43.600544 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:43.600544 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:43.600544 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:43.601465 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 12:01:43.601465 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 12:01:43.601465 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 12:01:43.601465 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 12:01:43.601465 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 12:01:43.601465 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 12:01:43.601465 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:01:43.601465 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:43.601465 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 339/678] 10.222.0.61 () {56 vars in 1111 bytes} [Fri Aug 26 12:01:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:43.601973 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93530cc-cadd-44ff-b9f6-b3ca6d0bc18f None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:43.611194 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f09f8b-00ba-45b0-a8d2-758fee4eb6ba None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:43.651287 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46db8eed-13bb-4c81-9359-18c43706218d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:43.651919 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46db8eed-13bb-4c81-9359-18c43706218d None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:43.652194 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46db8eed-13bb-4c81-9359-18c43706218d None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:43.652461 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46db8eed-13bb-4c81-9359-18c43706218d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:43.653834 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46db8eed-13bb-4c81-9359-18c43706218d None admin] RBAC: Adding query filter params (role.id=519347d2bd2c4dc1aadb0b9f8c3c8e3d, scope.project.id=ecd68423a83a427681d44dbf266fce6e, user.id=b0972dbd5cff4610b239326425e6351c) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:43.654280 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46db8eed-13bb-4c81-9359-18c43706218d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:43.655908 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46db8eed-13bb-4c81-9359-18c43706218d None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:43.663020 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 340/679] 10.222.0.61 () {56 vars in 1374 bytes} [Fri Aug 26 12:01:43 2022] GET /identity/v3/role_assignments?user.id=b0972dbd5cff4610b239326425e6351c&scope.project.id=ecd68423a83a427681d44dbf266fce6e&role.id=519347d2bd2c4dc1aadb0b9f8c3c8e3d => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:43.663688 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46db8eed-13bb-4c81-9359-18c43706218d None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:43.722220 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93530cc-cadd-44ff-b9f6-b3ca6d0bc18f None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:43.776624 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d96c39-0686-46e8-a23f-4b7664500c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:43.777991 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d96c39-0686-46e8-a23f-4b7664500c10 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:43.780333 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d96c39-0686-46e8-a23f-4b7664500c10 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:43.780794 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d96c39-0686-46e8-a23f-4b7664500c10 None admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:43.781643 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d96c39-0686-46e8-a23f-4b7664500c10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:43.783594 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d96c39-0686-46e8-a23f-4b7664500c10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:43.785188 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d96c39-0686-46e8-a23f-4b7664500c10 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:43.816567 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 340/680] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:01:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:43.824711 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d96c39-0686-46e8-a23f-4b7664500c10 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:43.826496 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46db8eed-13bb-4c81-9359-18c43706218d None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:43.893075 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:43.893270 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feb31a6-26f0-49a4-a94f-8b3d168bdd25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:43.893718 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb31a6-26f0-49a4-a94f-8b3d168bdd25 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:43.894167 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb31a6-26f0-49a4-a94f-8b3d168bdd25 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:43.894316 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb31a6-26f0-49a4-a94f-8b3d168bdd25 None admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:43.896037 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feb31a6-26f0-49a4-a94f-8b3d168bdd25 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:43.896706 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feb31a6-26f0-49a4-a94f-8b3d168bdd25 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:43.898152 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feb31a6-26f0-49a4-a94f-8b3d168bdd25 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:43.917443 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 341/681] 10.222.0.61 () {56 vars in 1148 bytes} [Fri Aug 26 12:01:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:43.923234 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feb31a6-26f0-49a4-a94f-8b3d168bdd25 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:43.942133 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:43.994382 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d96c39-0686-46e8-a23f-4b7664500c10 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:44.040367 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71a9d44-4f5e-42f0-9a7d-b8563e86fa1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:44.040913 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a9d44-4f5e-42f0-9a7d-b8563e86fa1d None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:44.041195 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a9d44-4f5e-42f0-9a7d-b8563e86fa1d None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:44.041460 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a9d44-4f5e-42f0-9a7d-b8563e86fa1d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:44.057220 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71a9d44-4f5e-42f0-9a7d-b8563e86fa1d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:44.057220 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71a9d44-4f5e-42f0-9a7d-b8563e86fa1d None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:44.058616 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feb31a6-26f0-49a4-a94f-8b3d168bdd25 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:44.069401 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-e71a9d44-4f5e-42f0-9a7d-b8563e86fa1d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 26 12:01:44.069401 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:44.069401 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:44.069401 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:44.069401 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:44.069401 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:44.069401 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:44.069401 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:44.069401 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:44.069401 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:44.069401 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:44.069401 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:44.069401 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 12:01:44.069401 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 12:01:44.069401 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 12:01:44.069401 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 12:01:44.070561 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:44.070561 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:44.070561 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 12:01:44.070561 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:01:44.070561 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 12:01:44.070561 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:01:44.070561 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:44.070561 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:44.070561 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:44.070561 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:44.070561 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:44.070561 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:44.070561 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:44.070561 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:44.070561 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:44.070561 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:44.070561 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:44.070561 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:44.071750 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:44.071750 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:44.071750 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 12:01:44.071750 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 12:01:44.071750 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 12:01:44.071750 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 12:01:44.071750 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 12:01:44.071750 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 12:01:44.071750 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 26 12:01:44.071750 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:44.079317 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 341/682] 10.222.0.61 () {56 vars in 1096 bytes} [Fri Aug 26 12:01:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 12:01:44.081359 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91d2245-cdfb-4810-b6fe-c1edbf9ddfa0 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:44.081790 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91d2245-cdfb-4810-b6fe-c1edbf9ddfa0 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:44.082219 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91d2245-cdfb-4810-b6fe-c1edbf9ddfa0 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:44.083973 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 342/683] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:01:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:01:44.085716 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91d2245-cdfb-4810-b6fe-c1edbf9ddfa0 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:44.105821 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a091fbf-3bc3-4fd9-9ee8-9a633ab4df05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:44.106561 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a091fbf-3bc3-4fd9-9ee8-9a633ab4df05 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:44.106847 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a091fbf-3bc3-4fd9-9ee8-9a633ab4df05 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:44.107188 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a091fbf-3bc3-4fd9-9ee8-9a633ab4df05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:44.108946 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a091fbf-3bc3-4fd9-9ee8-9a633ab4df05 None admin] RBAC: Adding query filter params (role.id=519347d2bd2c4dc1aadb0b9f8c3c8e3d, scope.project.id=ecd68423a83a427681d44dbf266fce6e, user.id=152fd7c708114f249b9c960165f0fdfe) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:44.109264 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a091fbf-3bc3-4fd9-9ee8-9a633ab4df05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:44.110710 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a091fbf-3bc3-4fd9-9ee8-9a633ab4df05 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:44.119617 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 342/684] 10.222.0.61 () {56 vars in 1374 bytes} [Fri Aug 26 12:01:43 2022] GET /identity/v3/role_assignments?user.id=152fd7c708114f249b9c960165f0fdfe&scope.project.id=ecd68423a83a427681d44dbf266fce6e&role.id=519347d2bd2c4dc1aadb0b9f8c3c8e3d => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:44.121107 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43888333-d738-470a-9212-7f622c491a25 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:44.121390 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43888333-d738-470a-9212-7f622c491a25 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:44.121902 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43888333-d738-470a-9212-7f622c491a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:44.146916 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-43888333-d738-470a-9212-7f622c491a25 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:01:44.148439 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-43888333-d738-470a-9212-7f622c491a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:01:44.190298 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:44.209953 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91d2245-cdfb-4810-b6fe-c1edbf9ddfa0 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:44.228808 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 343/685] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:01:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:01:44.248306 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1f55e7-d355-4bc5-9201-4592437f11d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:44.249391 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1f55e7-d355-4bc5-9201-4592437f11d4 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:44.249886 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1f55e7-d355-4bc5-9201-4592437f11d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:44.250375 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1f55e7-d355-4bc5-9201-4592437f11d4 None admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:44.251432 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1f55e7-d355-4bc5-9201-4592437f11d4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:44.251948 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1f55e7-d355-4bc5-9201-4592437f11d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:44.253628 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1f55e7-d355-4bc5-9201-4592437f11d4 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:44.279247 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 343/686] 10.222.0.61 () {56 vars in 1135 bytes} [Fri Aug 26 12:01:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:01:44.281011 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43888333-d738-470a-9212-7f622c491a25 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:44.293202 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:44.394759 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43888333-d738-470a-9212-7f622c491a25 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:44.435242 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ef9169-e9f6-482a-aca9-a4f6f165b921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:44.436203 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ef9169-e9f6-482a-aca9-a4f6f165b921 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:44.436677 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ef9169-e9f6-482a-aca9-a4f6f165b921 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:44.437185 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ef9169-e9f6-482a-aca9-a4f6f165b921 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:44.508339 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ef9169-e9f6-482a-aca9-a4f6f165b921 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:44.510121 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ef9169-e9f6-482a-aca9-a4f6f165b921 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:44.515108 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1f55e7-d355-4bc5-9201-4592437f11d4 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:44.521944 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:44.632232 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 344/687] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:01:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 12:01:44.639498 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ef9169-e9f6-482a-aca9-a4f6f165b921 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:44.656218 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1f55e7-d355-4bc5-9201-4592437f11d4 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:44.695402 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c332bd80-5df1-4029-bc94-79e8b87bd929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:44.696030 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c332bd80-5df1-4029-bc94-79e8b87bd929 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:44.696305 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c332bd80-5df1-4029-bc94-79e8b87bd929 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:44.696683 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c332bd80-5df1-4029-bc94-79e8b87bd929 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:44.707895 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c332bd80-5df1-4029-bc94-79e8b87bd929 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:01:44.713162 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-c332bd80-5df1-4029-bc94-79e8b87bd929 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 12:01:44.713162 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:44.713162 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:44.713162 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:44.713162 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:44.713162 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:44.713162 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:44.713162 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:44.713162 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:44.713162 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:44.713162 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:44.713162 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:44.713162 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:01:44.713162 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:01:44.713162 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:01:44.713162 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:01:44.714089 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:01:44.714089 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:01:44.714089 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:01:44.714089 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:01:44.714089 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:44.714089 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:44.714089 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:44.714089 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:44.714089 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:44.714089 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:44.714089 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:44.714089 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:44.714089 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:44.714089 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:44.714089 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:44.714089 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:44.714921 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:44.714921 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:44.714921 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:44.714921 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:44.714921 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:01:44.714921 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:01:44.714921 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:01:44.714921 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:01:44.714921 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:01:44.714921 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:01:44.714921 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 12:01:44.714921 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:44.714921 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 344/688] 10.222.0.61 () {56 vars in 1096 bytes} [Fri Aug 26 12:01:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:44.722380 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c332bd80-5df1-4029-bc94-79e8b87bd929 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:44.729798 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:44.755607 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ef9169-e9f6-482a-aca9-a4f6f165b921 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:44.766445 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:44.797311 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee219c9-bc18-41f2-9552-87b058658571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:44.798415 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee219c9-bc18-41f2-9552-87b058658571 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:44.798889 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee219c9-bc18-41f2-9552-87b058658571 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:44.799384 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee219c9-bc18-41f2-9552-87b058658571 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:44.811417 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee219c9-bc18-41f2-9552-87b058658571 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:44.813272 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee219c9-bc18-41f2-9552-87b058658571 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:44.830317 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-bee219c9-bc18-41f2-9552-87b058658571 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:44.830317 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:44.830317 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:44.830317 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:44.830317 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:44.830317 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:44.830317 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:44.830317 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:44.830317 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:44.830317 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:44.830317 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:44.830317 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:44.830317 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:01:44.830317 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:01:44.830317 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:01:44.830317 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:01:44.831318 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:44.831318 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:44.831318 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:44.831318 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:44.831318 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:44.831318 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:44.831318 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:44.831318 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:44.831318 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:44.831318 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:44.831318 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:44.831318 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:44.831318 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:44.831318 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:44.831318 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:44.831318 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:44.831318 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:01:44.832222 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:01:44.832222 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:44.832222 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:44.834335 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 345/689] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:01:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 12:01:44.839764 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c332bd80-5df1-4029-bc94-79e8b87bd929 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:44.840249 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee219c9-bc18-41f2-9552-87b058658571 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:44.878062 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114271e2-e75e-4443-b6c5-a65a14ef1ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:44.878812 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114271e2-e75e-4443-b6c5-a65a14ef1ce9 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:44.879252 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114271e2-e75e-4443-b6c5-a65a14ef1ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:44.879609 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114271e2-e75e-4443-b6c5-a65a14ef1ce9 None admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:44.880371 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114271e2-e75e-4443-b6c5-a65a14ef1ce9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:44.880771 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114271e2-e75e-4443-b6c5-a65a14ef1ce9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:44.882224 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114271e2-e75e-4443-b6c5-a65a14ef1ce9 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:44.892808 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 345/690] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:01:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:01:44.899234 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114271e2-e75e-4443-b6c5-a65a14ef1ce9 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:44.957507 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee219c9-bc18-41f2-9552-87b058658571 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:44.971787 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:44.993046 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bda720-8cc8-4edd-807f-3659e36ec024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:44.993762 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bda720-8cc8-4edd-807f-3659e36ec024 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:44.994119 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bda720-8cc8-4edd-807f-3659e36ec024 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:44.994439 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bda720-8cc8-4edd-807f-3659e36ec024 None admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:44.995148 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bda720-8cc8-4edd-807f-3659e36ec024 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:44.995820 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bda720-8cc8-4edd-807f-3659e36ec024 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:44.997615 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bda720-8cc8-4edd-807f-3659e36ec024 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:45.010262 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 346/691] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:01:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:01:45.017929 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bda720-8cc8-4edd-807f-3659e36ec024 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:45.020869 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114271e2-e75e-4443-b6c5-a65a14ef1ce9 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:45.062981 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:45.088871 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b088fdd6-cdf4-4ec6-af13-3ccbfbfd68ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:45.089901 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b088fdd6-cdf4-4ec6-af13-3ccbfbfd68ef None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:45.091943 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b088fdd6-cdf4-4ec6-af13-3ccbfbfd68ef None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:45.092440 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b088fdd6-cdf4-4ec6-af13-3ccbfbfd68ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:45.113352 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b088fdd6-cdf4-4ec6-af13-3ccbfbfd68ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:45.114823 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b088fdd6-cdf4-4ec6-af13-3ccbfbfd68ef None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:45.139271 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-b088fdd6-cdf4-4ec6-af13-3ccbfbfd68ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:45.139271 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:45.139271 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:45.139271 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:45.139271 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:45.139271 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:45.139271 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:45.139271 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:45.139271 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:45.139271 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:45.139271 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:45.139271 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:45.139271 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:01:45.139271 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:01:45.139271 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:01:45.139271 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:01:45.140367 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:45.140367 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:45.140367 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:45.140367 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:45.140367 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:45.140367 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:45.140367 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:45.140367 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:45.140367 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:45.140367 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:45.140367 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:45.140367 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:45.140367 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:45.140367 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:45.140367 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:45.140367 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:45.140367 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:01:45.141300 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:01:45.141300 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:45.141300 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:45.141300 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 346/692] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:01:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:45.152319 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b088fdd6-cdf4-4ec6-af13-3ccbfbfd68ef None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:45.189230 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bda720-8cc8-4edd-807f-3659e36ec024 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:45.228044 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef445d7-784e-4dad-a159-a2fc631ab970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:45.228673 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef445d7-784e-4dad-a159-a2fc631ab970 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:45.228949 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef445d7-784e-4dad-a159-a2fc631ab970 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:45.229218 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef445d7-784e-4dad-a159-a2fc631ab970 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:45.240420 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef445d7-784e-4dad-a159-a2fc631ab970 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:45.241858 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef445d7-784e-4dad-a159-a2fc631ab970 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:45.265005 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-2ef445d7-784e-4dad-a159-a2fc631ab970 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:01:45.265005 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:45.265005 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:45.265005 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:45.265005 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:45.265005 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:45.265005 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:45.265005 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:45.265005 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:45.265005 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:45.265005 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:45.265005 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:45.265005 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 12:01:45.265005 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 12:01:45.265005 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 12:01:45.265955 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 12:01:45.265955 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:45.265955 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:45.265955 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:45.265955 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:45.265955 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:45.265955 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:45.265955 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:45.265955 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:45.265955 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:45.265955 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:45.265955 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:45.265955 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:45.265955 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:45.265955 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:45.265955 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:45.265955 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:45.266863 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 12:01:45.266863 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 12:01:45.266863 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 12:01:45.266863 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 12:01:45.266863 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 12:01:45.266863 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 12:01:45.266863 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:01:45.266863 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:45.266863 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 347/693] 10.222.0.61 () {56 vars in 1111 bytes} [Fri Aug 26 12:01:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:45.272149 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef445d7-784e-4dad-a159-a2fc631ab970 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:45.345027 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b088fdd6-cdf4-4ec6-af13-3ccbfbfd68ef None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:45.376103 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:45.420118 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:45.420805 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a0f05-1f53-4252-ab44-28a53e095da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:45.421494 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a0f05-1f53-4252-ab44-28a53e095da6 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:45.421841 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a0f05-1f53-4252-ab44-28a53e095da6 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:45.422427 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a0f05-1f53-4252-ab44-28a53e095da6 None admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:45.422880 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3a0f05-1f53-4252-ab44-28a53e095da6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:45.423265 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3a0f05-1f53-4252-ab44-28a53e095da6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:45.424858 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3a0f05-1f53-4252-ab44-28a53e095da6 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:45.444271 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef445d7-784e-4dad-a159-a2fc631ab970 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:45.451228 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 347/694] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:01:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:45.468545 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a0f05-1f53-4252-ab44-28a53e095da6 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:45.492094 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136337bf-acaf-4c32-a151-0edc21a22569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:45.493020 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136337bf-acaf-4c32-a151-0edc21a22569 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:45.493471 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136337bf-acaf-4c32-a151-0edc21a22569 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:45.493911 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136337bf-acaf-4c32-a151-0edc21a22569 None admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:45.494768 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136337bf-acaf-4c32-a151-0edc21a22569 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:45.495291 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136337bf-acaf-4c32-a151-0edc21a22569 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:45.499529 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136337bf-acaf-4c32-a151-0edc21a22569 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:45.533889 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 348/695] 10.222.0.61 () {56 vars in 1148 bytes} [Fri Aug 26 12:01:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:45.538221 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136337bf-acaf-4c32-a151-0edc21a22569 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:45.682093 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136337bf-acaf-4c32-a151-0edc21a22569 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:45.725410 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e3f4a5-1690-4f05-8c79-0a0489e41b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:45.727549 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a0f05-1f53-4252-ab44-28a53e095da6 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:45.728688 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e3f4a5-1690-4f05-8c79-0a0489e41b65 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:45.729246 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e3f4a5-1690-4f05-8c79-0a0489e41b65 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:45.729694 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e3f4a5-1690-4f05-8c79-0a0489e41b65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:45.730759 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e3f4a5-1690-4f05-8c79-0a0489e41b65 None admin] RBAC: Adding query filter params (role.id=519347d2bd2c4dc1aadb0b9f8c3c8e3d, scope.project.id=ecd68423a83a427681d44dbf266fce6e, user.id=ead95a244d11432d9fe9f718e1a75f38) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:45.732749 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e3f4a5-1690-4f05-8c79-0a0489e41b65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:45.736077 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e3f4a5-1690-4f05-8c79-0a0489e41b65 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:45.752878 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 349/696] 10.222.0.61 () {56 vars in 1374 bytes} [Fri Aug 26 12:01:45 2022] GET /identity/v3/role_assignments?user.id=ead95a244d11432d9fe9f718e1a75f38&scope.project.id=ecd68423a83a427681d44dbf266fce6e&role.id=519347d2bd2c4dc1aadb0b9f8c3c8e3d => generated 608 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:45.790820 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b135506-8a1c-4ac3-9de9-85239eaaa48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:45.791500 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b135506-8a1c-4ac3-9de9-85239eaaa48c None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:45.791778 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b135506-8a1c-4ac3-9de9-85239eaaa48c None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:45.792049 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b135506-8a1c-4ac3-9de9-85239eaaa48c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:45.801326 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b135506-8a1c-4ac3-9de9-85239eaaa48c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:45.802862 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b135506-8a1c-4ac3-9de9-85239eaaa48c None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:45.823490 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-9b135506-8a1c-4ac3-9de9-85239eaaa48c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 12:01:45.823490 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:45.823490 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:45.823490 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:45.823490 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:45.823490 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:45.823490 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:45.823490 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:45.823490 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:45.823490 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:45.823490 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:45.823490 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:45.823490 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 12:01:45.823490 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 12:01:45.823490 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 12:01:45.823490 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 12:01:45.824492 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:45.824492 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:45.824492 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 12:01:45.824492 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:01:45.824492 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 12:01:45.824492 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:01:45.824492 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:45.824492 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:45.824492 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:45.824492 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:45.824492 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:45.824492 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:45.824492 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:45.824492 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:45.824492 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:45.824492 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:45.824492 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:45.824492 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:45.825484 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:45.825484 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:45.825484 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 12:01:45.825484 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 12:01:45.825484 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 12:01:45.825484 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 12:01:45.825484 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 12:01:45.825484 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 12:01:45.825484 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 12:01:45.825484 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:45.825484 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 348/697] 10.222.0.61 () {56 vars in 1093 bytes} [Fri Aug 26 12:01:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 12:01:45.832377 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e3f4a5-1690-4f05-8c79-0a0489e41b65 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:45.834618 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a936f205-3911-4432-aace-5a967cb5bbc8 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:45.834941 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a936f205-3911-4432-aace-5a967cb5bbc8 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:45.835239 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a936f205-3911-4432-aace-5a967cb5bbc8 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:45.837225 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 349/698] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:01:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:01:45.840166 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:45.843896 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40eda19-c9b9-4730-91db-3ae3708adaad None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:45.844203 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40eda19-c9b9-4730-91db-3ae3708adaad None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:45.844453 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40eda19-c9b9-4730-91db-3ae3708adaad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:45.884355 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-e40eda19-c9b9-4730-91db-3ae3708adaad None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:01:45.886882 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-e40eda19-c9b9-4730-91db-3ae3708adaad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:01:45.937499 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:45.995229 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 350/699] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:01:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 12:01:46.005408 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e3f4a5-1690-4f05-8c79-0a0489e41b65 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:46.019437 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8742b9b3-3a4c-4297-96d5-2f88053e1bb8 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:46.019719 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8742b9b3-3a4c-4297-96d5-2f88053e1bb8 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:46.019994 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8742b9b3-3a4c-4297-96d5-2f88053e1bb8 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:46.021480 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 351/700] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:01:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 12:01:46.026599 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b6935d-f11c-4d56-8477-985ca652e9c5 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:46.026878 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b6935d-f11c-4d56-8477-985ca652e9c5 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:46.027151 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b6935d-f11c-4d56-8477-985ca652e9c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:46.050287 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-b4b6935d-f11c-4d56-8477-985ca652e9c5 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:01:46.051507 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-b4b6935d-f11c-4d56-8477-985ca652e9c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:01:46.060593 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52147720-7ba7-4145-92d9-6804b88f78bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:46.061639 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52147720-7ba7-4145-92d9-6804b88f78bb None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:46.062232 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52147720-7ba7-4145-92d9-6804b88f78bb None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:46.062576 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52147720-7ba7-4145-92d9-6804b88f78bb None admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:46.063437 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52147720-7ba7-4145-92d9-6804b88f78bb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:46.063820 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52147720-7ba7-4145-92d9-6804b88f78bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:46.065554 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52147720-7ba7-4145-92d9-6804b88f78bb None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:46.086327 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 350/701] 10.222.0.61 () {56 vars in 1133 bytes} [Fri Aug 26 12:01:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:46.091868 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52147720-7ba7-4145-92d9-6804b88f78bb None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:46.134047 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 352/702] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:01:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 12:01:46.163047 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:46.208111 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52147720-7ba7-4145-92d9-6804b88f78bb None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:46.243779 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a79bdc-f840-4bdf-abdf-7431ddadba5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:46.244381 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a79bdc-f840-4bdf-abdf-7431ddadba5e None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:46.244661 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a79bdc-f840-4bdf-abdf-7431ddadba5e None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:46.244971 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a79bdc-f840-4bdf-abdf-7431ddadba5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:46.314783 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a79bdc-f840-4bdf-abdf-7431ddadba5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:46.316252 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a79bdc-f840-4bdf-abdf-7431ddadba5e None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:46.356115 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b6935d-f11c-4d56-8477-985ca652e9c5 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:46.379128 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:46.423062 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 351/703] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:01:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 12:01:46.428935 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a79bdc-f840-4bdf-abdf-7431ddadba5e None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:46.463672 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:46.472457 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b6935d-f11c-4d56-8477-985ca652e9c5 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:46.513259 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf4fdfb-15d2-4e35-ac06-0bdb2ba4c67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:46.514292 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf4fdfb-15d2-4e35-ac06-0bdb2ba4c67b None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:46.514778 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf4fdfb-15d2-4e35-ac06-0bdb2ba4c67b None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:46.515392 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf4fdfb-15d2-4e35-ac06-0bdb2ba4c67b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:46.516163 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf4fdfb-15d2-4e35-ac06-0bdb2ba4c67b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:46.517719 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf4fdfb-15d2-4e35-ac06-0bdb2ba4c67b None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:46.530120 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-6cf4fdfb-15d2-4e35-ac06-0bdb2ba4c67b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 12:01:46.530120 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:46.530120 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 12:01:46.530120 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 12:01:46.530120 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 12:01:46.530120 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 12:01:46.530120 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 12:01:46.530120 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 12:01:46.530120 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 12:01:46.530120 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:46.530120 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 12:01:46.530120 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:46.530120 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:46.530120 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:46.530120 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:46.530120 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:46.530120 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:46.531373 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:46.531373 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:46.531373 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:46.531373 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:46.531373 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:46.531373 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:46.531373 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 12:01:46.531373 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 12:01:46.531373 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 12:01:46.531373 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 12:01:46.531373 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:46.531373 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:46.531373 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:46.531373 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:46.531373 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:46.531373 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:46.532230 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:46.532230 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:46.532230 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:46.532230 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:46.532230 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:46.532230 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:46.532230 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:46.532230 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:46.532230 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:46.532230 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:46.532230 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 12:01:46.532230 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 12:01:46.532230 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 12:01:46.532230 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:46.539295 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 353/704] 10.222.0.61 () {56 vars in 1129 bytes} [Fri Aug 26 12:01:46 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:46.540399 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf4fdfb-15d2-4e35-ac06-0bdb2ba4c67b None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:46.553025 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a79bdc-f840-4bdf-abdf-7431ddadba5e None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:46.599899 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92502c4-2a0b-4758-bcb1-ff62c287805a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:46.600529 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92502c4-2a0b-4758-bcb1-ff62c287805a None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:46.600803 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92502c4-2a0b-4758-bcb1-ff62c287805a None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:46.601069 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92502c4-2a0b-4758-bcb1-ff62c287805a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:46.614944 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92502c4-2a0b-4758-bcb1-ff62c287805a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:46.616678 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92502c4-2a0b-4758-bcb1-ff62c287805a None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:46.636554 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-d92502c4-2a0b-4758-bcb1-ff62c287805a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:46.636554 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:46.636554 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:46.636554 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:46.636554 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:46.636554 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:46.636554 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:46.636554 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:46.636554 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:46.636554 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:46.636554 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:46.636554 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:46.636554 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:01:46.636554 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:01:46.636554 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:01:46.636554 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:01:46.638127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:46.638127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:46.638127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:46.638127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:46.638127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:46.638127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:46.638127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:46.638127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:46.638127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:46.638127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:46.638127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:46.638127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:46.638127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:46.638127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:46.638127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:46.638127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:46.638127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:01:46.639058 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:01:46.639058 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:46.639058 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:46.639058 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 352/705] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:01:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 12:01:46.639818 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92502c4-2a0b-4758-bcb1-ff62c287805a None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:46.659667 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:46.664802 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf4fdfb-15d2-4e35-ac06-0bdb2ba4c67b None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:46.715925 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd1c887-117e-4dc2-aad3-aeaec7d564fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:46.716452 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd1c887-117e-4dc2-aad3-aeaec7d564fa None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:46.716784 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd1c887-117e-4dc2-aad3-aeaec7d564fa None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:46.716992 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd1c887-117e-4dc2-aad3-aeaec7d564fa None admin] PATH_INFO: `/v3/services/network` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:46.717872 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd1c887-117e-4dc2-aad3-aeaec7d564fa None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:46.719946 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd1c887-117e-4dc2-aad3-aeaec7d564fa None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:46.736502 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-4fd1c887-117e-4dc2-aad3-aeaec7d564fa None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 12:01:46.736502 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:46.736502 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 12:01:46.736502 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 12:01:46.736502 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 12:01:46.736502 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 12:01:46.736502 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 12:01:46.736502 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 12:01:46.736502 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 12:01:46.736502 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:46.736502 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 12:01:46.736502 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:46.736502 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:46.736502 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:46.736502 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:46.736502 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:46.736502 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:46.737992 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:46.737992 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:46.737992 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:46.737992 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:46.737992 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:46.737992 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:46.737992 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 12:01:46.737992 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 12:01:46.737992 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 12:01:46.737992 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 12:01:46.737992 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:46.737992 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:46.737992 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:46.737992 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:46.737992 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:46.737992 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:46.738866 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:46.738866 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:46.738866 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:46.738866 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:46.738866 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:46.738866 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:46.738866 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:46.738866 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:46.738866 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:46.738866 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:46.738866 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 12:01:46.738866 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 12:01:46.738866 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 12:01:46.738866 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:46.738866 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 354/706] 10.222.0.61 () {56 vars in 1111 bytes} [Fri Aug 26 12:01:46 2022] GET /identity/v3/services/network => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:46.740220 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd1c887-117e-4dc2-aad3-aeaec7d564fa None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:46.762731 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92502c4-2a0b-4758-bcb1-ff62c287805a None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:46.777259 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:46.807116 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baedac54-0d45-4da8-a3fb-1574b3a01dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:46.807748 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baedac54-0d45-4da8-a3fb-1574b3a01dbb None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:46.808023 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baedac54-0d45-4da8-a3fb-1574b3a01dbb None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:46.808289 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baedac54-0d45-4da8-a3fb-1574b3a01dbb None admin] PATH_INFO: `/v3/services` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:46.809339 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baedac54-0d45-4da8-a3fb-1574b3a01dbb None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:46.810275 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baedac54-0d45-4da8-a3fb-1574b3a01dbb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:46.811953 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baedac54-0d45-4da8-a3fb-1574b3a01dbb None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:46.822709 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 353/707] 10.222.0.61 () {56 vars in 1124 bytes} [Fri Aug 26 12:01:46 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:46.822709 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baedac54-0d45-4da8-a3fb-1574b3a01dbb None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:46.866272 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd1c887-117e-4dc2-aad3-aeaec7d564fa None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:46.911164 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:46.913257 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc5acef-3fb1-4280-bb6b-e736199e7ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:46.914019 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc5acef-3fb1-4280-bb6b-e736199e7ae4 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:46.914368 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc5acef-3fb1-4280-bb6b-e736199e7ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:46.914642 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc5acef-3fb1-4280-bb6b-e736199e7ae4 None admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:46.915383 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc5acef-3fb1-4280-bb6b-e736199e7ae4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:46.915798 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc5acef-3fb1-4280-bb6b-e736199e7ae4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:46.918055 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc5acef-3fb1-4280-bb6b-e736199e7ae4 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:46.940044 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 355/708] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:01:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:46.940044 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc5acef-3fb1-4280-bb6b-e736199e7ae4 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:46.943787 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baedac54-0d45-4da8-a3fb-1574b3a01dbb None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:46.981310 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79b21c7-dab5-46c3-8b16-6e12401e2b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:46.982845 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79b21c7-dab5-46c3-8b16-6e12401e2b68 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:46.983379 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79b21c7-dab5-46c3-8b16-6e12401e2b68 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:46.983886 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79b21c7-dab5-46c3-8b16-6e12401e2b68 None admin] PATH_INFO: `/v3/services` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:46.984707 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79b21c7-dab5-46c3-8b16-6e12401e2b68 None admin] RBAC: Adding query filter params (name=network) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:46.985222 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79b21c7-dab5-46c3-8b16-6e12401e2b68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:46.986881 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79b21c7-dab5-46c3-8b16-6e12401e2b68 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:46.992325 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 354/709] 10.222.0.61 () {56 vars in 1112 bytes} [Fri Aug 26 12:01:46 2022] GET /identity/v3/services?name=network => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:46.993369 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79b21c7-dab5-46c3-8b16-6e12401e2b68 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:47.046684 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:47.068253 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc5acef-3fb1-4280-bb6b-e736199e7ae4 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:47.101471 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79b21c7-dab5-46c3-8b16-6e12401e2b68 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:47.106080 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98187a8b-5253-4d2b-852d-c751dd27a38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:47.106788 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98187a8b-5253-4d2b-852d-c751dd27a38b None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:47.107070 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98187a8b-5253-4d2b-852d-c751dd27a38b None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:47.107378 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98187a8b-5253-4d2b-852d-c751dd27a38b None admin] PATH_INFO: `/v3/services` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:47.108012 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98187a8b-5253-4d2b-852d-c751dd27a38b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:47.108316 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98187a8b-5253-4d2b-852d-c751dd27a38b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:47.109751 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98187a8b-5253-4d2b-852d-c751dd27a38b None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:47.115750 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 356/710] 10.222.0.61 () {56 vars in 1124 bytes} [Fri Aug 26 12:01:46 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:47.116533 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98187a8b-5253-4d2b-852d-c751dd27a38b None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:47.156911 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6caa2d-e9f4-41b4-9195-f7b8b8b33e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:47.157935 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6caa2d-e9f4-41b4-9195-f7b8b8b33e34 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:47.158727 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6caa2d-e9f4-41b4-9195-f7b8b8b33e34 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:47.159135 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6caa2d-e9f4-41b4-9195-f7b8b8b33e34 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:47.172140 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6caa2d-e9f4-41b4-9195-f7b8b8b33e34 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:47.174138 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6caa2d-e9f4-41b4-9195-f7b8b8b33e34 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:47.192190 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-0a6caa2d-e9f4-41b4-9195-f7b8b8b33e34 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:01:47.192190 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:47.192190 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:47.192190 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:47.192190 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:47.192190 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:47.192190 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:47.192190 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:47.192190 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:47.192190 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:47.192190 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:47.192190 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:47.192190 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 12:01:47.192190 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 12:01:47.192190 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 12:01:47.193366 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 12:01:47.193366 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:47.193366 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:47.193366 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:47.193366 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:47.193366 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:47.193366 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:47.193366 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:47.193366 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:47.193366 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:47.193366 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:47.193366 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:47.193366 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:47.193366 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:47.193366 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:47.193366 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:47.193366 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:47.194457 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 12:01:47.194457 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 12:01:47.194457 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 12:01:47.194457 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 12:01:47.194457 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 12:01:47.194457 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 12:01:47.194457 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:01:47.194457 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:47.199441 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 355/711] 10.222.0.61 () {56 vars in 1111 bytes} [Fri Aug 26 12:01:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:47.202075 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1933f389-32b2-45fa-b8ec-0163fa32e2a7 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:47.202367 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1933f389-32b2-45fa-b8ec-0163fa32e2a7 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:47.202645 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1933f389-32b2-45fa-b8ec-0163fa32e2a7 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:47.204690 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 356/712] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:01:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:01:47.205131 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1933f389-32b2-45fa-b8ec-0163fa32e2a7 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:47.208256 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:47.265970 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98187a8b-5253-4d2b-852d-c751dd27a38b None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:47.326203 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31abdf6-ff43-43c5-89d3-15263dd76c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:47.327438 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31abdf6-ff43-43c5-89d3-15263dd76c1c None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:47.327928 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31abdf6-ff43-43c5-89d3-15263dd76c1c None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:47.328409 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31abdf6-ff43-43c5-89d3-15263dd76c1c None admin] PATH_INFO: `/v3/services` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:47.328537 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1933f389-32b2-45fa-b8ec-0163fa32e2a7 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:47.329374 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31abdf6-ff43-43c5-89d3-15263dd76c1c None admin] RBAC: Adding query filter params (type=network) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:47.330132 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31abdf6-ff43-43c5-89d3-15263dd76c1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:47.332432 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31abdf6-ff43-43c5-89d3-15263dd76c1c None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:47.339692 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 357/713] 10.222.0.61 () {56 vars in 1112 bytes} [Fri Aug 26 12:01:47 2022] GET /identity/v3/services?type=network => generated 124 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:47.343447 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa9f9f-c7c4-4df5-8368-9a89e3068fde None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:47.344233 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa9f9f-c7c4-4df5-8368-9a89e3068fde None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:47.344755 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa9f9f-c7c4-4df5-8368-9a89e3068fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:47.369732 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6d214f-5175-4361-bea1-5b6f127ba376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:47.370324 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-d5aa9f9f-c7c4-4df5-8368-9a89e3068fde None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:01:47.370451 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d214f-5175-4361-bea1-5b6f127ba376 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:47.370798 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d214f-5175-4361-bea1-5b6f127ba376 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:47.371233 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d214f-5175-4361-bea1-5b6f127ba376 None admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:47.372231 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6d214f-5175-4361-bea1-5b6f127ba376 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:47.372782 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6d214f-5175-4361-bea1-5b6f127ba376 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:47.373081 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-d5aa9f9f-c7c4-4df5-8368-9a89e3068fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:01:47.374618 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6d214f-5175-4361-bea1-5b6f127ba376 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:47.383700 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:47.391436 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 357/714] 10.222.0.61 () {56 vars in 1148 bytes} [Fri Aug 26 12:01:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:47.402327 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6d214f-5175-4361-bea1-5b6f127ba376 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:47.514588 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 358/715] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:01:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:01:47.540587 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:47.578327 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6d214f-5175-4361-bea1-5b6f127ba376 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:47.615488 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304d01fd-2d98-460f-9ce6-0ac96e3064c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:47.616394 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304d01fd-2d98-460f-9ce6-0ac96e3064c1 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:47.616872 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304d01fd-2d98-460f-9ce6-0ac96e3064c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:47.617342 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304d01fd-2d98-460f-9ce6-0ac96e3064c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:47.618306 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304d01fd-2d98-460f-9ce6-0ac96e3064c1 None admin] RBAC: Adding query filter params (role.id=70ec1ba484714c8ca3adb8fe5845107c, scope.project.id=ecd68423a83a427681d44dbf266fce6e, user.id=7aacd7d6d17e4f10ba5c9107a12beafa) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:47.618837 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304d01fd-2d98-460f-9ce6-0ac96e3064c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:47.620490 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304d01fd-2d98-460f-9ce6-0ac96e3064c1 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:47.627514 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 358/716] 10.222.0.61 () {56 vars in 1374 bytes} [Fri Aug 26 12:01:47 2022] GET /identity/v3/role_assignments?user.id=7aacd7d6d17e4f10ba5c9107a12beafa&scope.project.id=ecd68423a83a427681d44dbf266fce6e&role.id=70ec1ba484714c8ca3adb8fe5845107c => generated 259 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:47.823936 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aa9f9f-c7c4-4df5-8368-9a89e3068fde None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:47.950209 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aa9f9f-c7c4-4df5-8368-9a89e3068fde None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:48.004815 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fe158a-ad23-483d-b890-5a4184237f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:48.005823 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fe158a-ad23-483d-b890-5a4184237f9a None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:48.006332 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fe158a-ad23-483d-b890-5a4184237f9a None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:48.006802 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fe158a-ad23-483d-b890-5a4184237f9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:48.020550 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fe158a-ad23-483d-b890-5a4184237f9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:48.022287 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fe158a-ad23-483d-b890-5a4184237f9a None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:48.024294 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:48.043516 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-30fe158a-ad23-483d-b890-5a4184237f9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:48.043516 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:48.043516 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:48.043516 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:48.043516 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:48.043516 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:48.043516 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:48.043516 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:48.043516 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:48.043516 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:48.043516 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:48.043516 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:48.043516 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:01:48.043516 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:01:48.043516 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:01:48.043516 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:01:48.045113 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:48.045113 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:48.045113 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:48.045113 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:48.045113 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:48.045113 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:48.045113 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:48.045113 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:48.045113 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:48.045113 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:48.045113 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:48.045113 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:48.045113 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:48.045113 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:48.045113 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:48.045113 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:48.045113 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:01:48.046019 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:01:48.046019 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:48.046019 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:48.051736 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 359/717] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:01:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:48.057134 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304d01fd-2d98-460f-9ce6-0ac96e3064c1 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:48.183298 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304d01fd-2d98-460f-9ce6-0ac96e3064c1 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:48.211603 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:48.218940 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3dccd7-9177-4ba1-ae88-7bd524214f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:48.219576 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3dccd7-9177-4ba1-ae88-7bd524214f9e None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:48.219854 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3dccd7-9177-4ba1-ae88-7bd524214f9e None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:48.220123 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3dccd7-9177-4ba1-ae88-7bd524214f9e None admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:48.220760 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3dccd7-9177-4ba1-ae88-7bd524214f9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:48.221059 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3dccd7-9177-4ba1-ae88-7bd524214f9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:48.222422 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3dccd7-9177-4ba1-ae88-7bd524214f9e None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:48.235212 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 359/718] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:01:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:48.240715 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fe158a-ad23-483d-b890-5a4184237f9a None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:48.344129 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fe158a-ad23-483d-b890-5a4184237f9a None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:48.359643 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c851135-ec1e-47dc-878c-7570ddd071a9 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:48.359952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c851135-ec1e-47dc-878c-7570ddd071a9 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:48.360213 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c851135-ec1e-47dc-878c-7570ddd071a9 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:48.361781 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 360/719] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:01:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 12:01:48.366773 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894158fd-f709-4f1a-a4fd-6e9672bc6d64 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:48.367060 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894158fd-f709-4f1a-a4fd-6e9672bc6d64 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:48.367365 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894158fd-f709-4f1a-a4fd-6e9672bc6d64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:48.381466 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9d6775-090d-42f2-87f4-c61a7f895cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:48.382433 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9d6775-090d-42f2-87f4-c61a7f895cf3 None admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:48.382936 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9d6775-090d-42f2-87f4-c61a7f895cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:48.383690 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9d6775-090d-42f2-87f4-c61a7f895cf3 None admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:48.384541 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9d6775-090d-42f2-87f4-c61a7f895cf3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:48.386234 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-894158fd-f709-4f1a-a4fd-6e9672bc6d64 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:01:48.386496 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9d6775-090d-42f2-87f4-c61a7f895cf3 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:48.387454 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-894158fd-f709-4f1a-a4fd-6e9672bc6d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:01:48.413186 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 360/720] 10.222.0.61 () {60 vars in 1132 bytes} [Fri Aug 26 12:01:48 2022] POST /identity/v3/users => generated 254 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:01:48.479499 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 361/721] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:01:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 12:01:48.821576 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9d6775-090d-42f2-87f4-c61a7f895cf3 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:48.874285 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:48.925863 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9d6775-090d-42f2-87f4-c61a7f895cf3 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:48.961078 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a974247-9b8d-4ffd-9442-a53616d2e6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:48.961715 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a974247-9b8d-4ffd-9442-a53616d2e6b4 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:48.961994 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a974247-9b8d-4ffd-9442-a53616d2e6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:48.962264 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a974247-9b8d-4ffd-9442-a53616d2e6b4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:48.973623 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a974247-9b8d-4ffd-9442-a53616d2e6b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:01:48.978807 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-1a974247-9b8d-4ffd-9442-a53616d2e6b4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 12:01:48.978807 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:48.978807 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:48.978807 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:48.978807 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:48.978807 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:48.978807 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:48.978807 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:48.978807 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:48.978807 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:48.978807 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:48.978807 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:48.978807 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:01:48.978807 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:01:48.978807 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:01:48.978807 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:01:48.979775 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:01:48.979775 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:01:48.979775 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:01:48.979775 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:01:48.979775 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:48.979775 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:48.979775 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:48.979775 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:48.979775 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:48.979775 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:48.979775 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:48.979775 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:48.979775 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:48.979775 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:48.979775 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:48.979775 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:48.980980 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:48.980980 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:48.980980 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:48.980980 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:48.980980 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:01:48.980980 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:01:48.980980 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:01:48.980980 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:01:48.980980 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:01:48.980980 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:01:48.980980 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 12:01:48.980980 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:48.980980 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 361/722] 10.222.0.61 () {56 vars in 1096 bytes} [Fri Aug 26 12:01:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:48.986227 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894158fd-f709-4f1a-a4fd-6e9672bc6d64 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:49.104245 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894158fd-f709-4f1a-a4fd-6e9672bc6d64 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:49.125874 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:49.145879 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8042e2f0-ad18-4309-8185-c437e52ce375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:49.146483 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8042e2f0-ad18-4309-8185-c437e52ce375 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:49.146760 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8042e2f0-ad18-4309-8185-c437e52ce375 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:49.147033 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8042e2f0-ad18-4309-8185-c437e52ce375 None admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:49.147719 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8042e2f0-ad18-4309-8185-c437e52ce375 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:49.148021 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8042e2f0-ad18-4309-8185-c437e52ce375 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:49.149380 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8042e2f0-ad18-4309-8185-c437e52ce375 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:49.162679 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 362/723] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:01:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:49.168379 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a974247-9b8d-4ffd-9442-a53616d2e6b4 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:49.293385 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a974247-9b8d-4ffd-9442-a53616d2e6b4 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:49.308933 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:49.334155 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dda010f-7aa7-4fe1-ac80-78cb1c4d23d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:49.334783 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dda010f-7aa7-4fe1-ac80-78cb1c4d23d8 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:49.335058 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dda010f-7aa7-4fe1-ac80-78cb1c4d23d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:49.335355 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dda010f-7aa7-4fe1-ac80-78cb1c4d23d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:49.350903 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dda010f-7aa7-4fe1-ac80-78cb1c4d23d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:49.352443 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dda010f-7aa7-4fe1-ac80-78cb1c4d23d8 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:49.369988 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-9dda010f-7aa7-4fe1-ac80-78cb1c4d23d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:49.369988 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:49.369988 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:49.369988 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:49.369988 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:49.369988 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:49.369988 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:49.369988 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:49.369988 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:49.369988 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:49.369988 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:49.369988 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:49.369988 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:01:49.369988 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:01:49.369988 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:01:49.369988 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:01:49.370968 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:49.370968 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:49.370968 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:49.370968 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:49.370968 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:49.370968 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:49.370968 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:49.370968 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:49.370968 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:49.370968 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:49.370968 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:49.370968 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:49.370968 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:49.370968 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:49.370968 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:49.370968 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:49.370968 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:01:49.372589 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:01:49.372589 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:49.372589 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:49.372589 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 362/724] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:01:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:49.377066 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8042e2f0-ad18-4309-8185-c437e52ce375 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:49.497577 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a1067-f175-4f01-946a-22a55bf87291 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:49.498052 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a1067-f175-4f01-946a-22a55bf87291 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:49.498330 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a1067-f175-4f01-946a-22a55bf87291 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:49.499728 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 363/725] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:01:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 12:01:49.503673 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8042e2f0-ad18-4309-8185-c437e52ce375 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:49.508448 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b213001-25fa-4281-b72b-7ac7daa144d4 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:49.508742 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b213001-25fa-4281-b72b-7ac7daa144d4 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:49.509021 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b213001-25fa-4281-b72b-7ac7daa144d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:49.529914 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-1b213001-25fa-4281-b72b-7ac7daa144d4 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:01:49.531646 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-1b213001-25fa-4281-b72b-7ac7daa144d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:01:49.568314 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da1b3e-4f79-4e41-bdb3-5119201d083b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:49.569437 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da1b3e-4f79-4e41-bdb3-5119201d083b None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:49.570021 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da1b3e-4f79-4e41-bdb3-5119201d083b None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:49.570457 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da1b3e-4f79-4e41-bdb3-5119201d083b None admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:49.571301 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0da1b3e-4f79-4e41-bdb3-5119201d083b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:49.571805 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0da1b3e-4f79-4e41-bdb3-5119201d083b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:49.573431 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0da1b3e-4f79-4e41-bdb3-5119201d083b None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:49.601105 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 363/726] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:01:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:49.609311 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da1b3e-4f79-4e41-bdb3-5119201d083b None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:49.657350 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 364/727] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:01:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:01:49.683569 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:49.740618 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d423cdd-fd89-4858-a619-a8558890609a None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:49.740920 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d423cdd-fd89-4858-a619-a8558890609a None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:49.741270 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d423cdd-fd89-4858-a619-a8558890609a None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:49.742838 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 365/728] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:01:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 12:01:49.746751 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da1b3e-4f79-4e41-bdb3-5119201d083b None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:49.756936 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd1977a-7fb7-4582-a042-d747daa327f3 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:49.757212 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd1977a-7fb7-4582-a042-d747daa327f3 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:49.757487 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd1977a-7fb7-4582-a042-d747daa327f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:49.788602 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-dfd1977a-7fb7-4582-a042-d747daa327f3 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:01:49.789801 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-dfd1977a-7fb7-4582-a042-d747daa327f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:01:49.817307 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de9153c-63f5-42c2-b6ea-bba47f902395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:49.817945 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9153c-63f5-42c2-b6ea-bba47f902395 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:49.818221 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9153c-63f5-42c2-b6ea-bba47f902395 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:49.818495 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9153c-63f5-42c2-b6ea-bba47f902395 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:49.834152 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de9153c-63f5-42c2-b6ea-bba47f902395 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:49.836740 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de9153c-63f5-42c2-b6ea-bba47f902395 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:49.849744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-4de9153c-63f5-42c2-b6ea-bba47f902395 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 26 12:01:49.849744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:49.849744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:49.849744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:49.849744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:49.849744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:49.849744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:49.849744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:49.849744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:49.849744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:49.849744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:49.849744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:49.849744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 12:01:49.849744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 12:01:49.849744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 12:01:49.849744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 12:01:49.850733 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:49.850733 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:49.850733 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 12:01:49.850733 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:01:49.850733 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 12:01:49.850733 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:01:49.850733 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:49.850733 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:49.850733 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:49.850733 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:49.850733 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:49.850733 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:49.850733 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:49.850733 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:49.850733 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:49.850733 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:49.850733 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:49.850733 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:49.851774 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:49.851774 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:49.851774 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 12:01:49.851774 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 12:01:49.851774 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 12:01:49.851774 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 12:01:49.851774 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 12:01:49.851774 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 12:01:49.851774 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 26 12:01:49.851774 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:49.851774 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 364/729] 10.222.0.61 () {56 vars in 1096 bytes} [Fri Aug 26 12:01:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 12:01:49.857228 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de9153c-63f5-42c2-b6ea-bba47f902395 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:49.941882 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 366/730] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:01:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 26 12:01:49.959587 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:49.962577 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de9153c-63f5-42c2-b6ea-bba47f902395 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:49.999077 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc21d12f-e9b7-44b2-96b3-0e3d7d3acc1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:49.999718 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc21d12f-e9b7-44b2-96b3-0e3d7d3acc1f None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:49.999999 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc21d12f-e9b7-44b2-96b3-0e3d7d3acc1f None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:50.000271 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc21d12f-e9b7-44b2-96b3-0e3d7d3acc1f None admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:50.000991 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc21d12f-e9b7-44b2-96b3-0e3d7d3acc1f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:50.001292 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc21d12f-e9b7-44b2-96b3-0e3d7d3acc1f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:50.002712 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc21d12f-e9b7-44b2-96b3-0e3d7d3acc1f None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:50.021331 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 365/731] 10.222.0.61 () {56 vars in 1135 bytes} [Fri Aug 26 12:01:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:01:50.026907 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd1977a-7fb7-4582-a042-d747daa327f3 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:50.064670 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc21d12f-e9b7-44b2-96b3-0e3d7d3acc1f None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:50.074019 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:50.174451 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc21d12f-e9b7-44b2-96b3-0e3d7d3acc1f None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:50.205393 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd1977a-7fb7-4582-a042-d747daa327f3 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:50.211393 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45de7cc2-e22c-46d6-8e7e-cdfb301ebdbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:50.212268 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45de7cc2-e22c-46d6-8e7e-cdfb301ebdbd None admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:50.212748 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45de7cc2-e22c-46d6-8e7e-cdfb301ebdbd None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:50.213272 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45de7cc2-e22c-46d6-8e7e-cdfb301ebdbd None admin] PATH_INFO: `/v3/services` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:50.214126 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45de7cc2-e22c-46d6-8e7e-cdfb301ebdbd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:50.216380 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45de7cc2-e22c-46d6-8e7e-cdfb301ebdbd None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:50.240303 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 366/732] 10.222.0.61 () {60 vars in 1141 bytes} [Fri Aug 26 12:01:50 2022] POST /identity/v3/services => generated 251 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 12:01:50.247356 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03dfe9b-cd90-417f-94e9-a38414e73a3f None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:50.247356 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03dfe9b-cd90-417f-94e9-a38414e73a3f None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:50.247356 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03dfe9b-cd90-417f-94e9-a38414e73a3f None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:50.248680 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 367/733] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:01:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:01:50.257049 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede16384-10d5-4c1e-a1f5-9256db0eed1c None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:50.257404 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede16384-10d5-4c1e-a1f5-9256db0eed1c None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:50.257600 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede16384-10d5-4c1e-a1f5-9256db0eed1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:50.269453 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5e2e4a-11cc-4216-8d9d-bea9d748d793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:50.270084 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5e2e4a-11cc-4216-8d9d-bea9d748d793 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:50.270359 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5e2e4a-11cc-4216-8d9d-bea9d748d793 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:50.270629 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5e2e4a-11cc-4216-8d9d-bea9d748d793 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:50.283146 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-ede16384-10d5-4c1e-a1f5-9256db0eed1c None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:01:50.284522 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-ede16384-10d5-4c1e-a1f5-9256db0eed1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:01:50.298061 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:50.356141 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5e2e4a-11cc-4216-8d9d-bea9d748d793 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:50.358933 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5e2e4a-11cc-4216-8d9d-bea9d748d793 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:50.373682 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 368/734] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:01:50 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 26 12:01:50.374341 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede16384-10d5-4c1e-a1f5-9256db0eed1c None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:50.385451 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:50.481178 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 367/735] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:01:50 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 26 12:01:50.486365 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5e2e4a-11cc-4216-8d9d-bea9d748d793 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:50.509713 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede16384-10d5-4c1e-a1f5-9256db0eed1c None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:50.556120 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79895d0-5a51-4fc3-b022-599bb9ba3a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:50.556969 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79895d0-5a51-4fc3-b022-599bb9ba3a3f None admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:50.557275 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79895d0-5a51-4fc3-b022-599bb9ba3a3f None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:50.557549 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79895d0-5a51-4fc3-b022-599bb9ba3a3f None admin] PATH_INFO: `/v3/services` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:50.558197 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79895d0-5a51-4fc3-b022-599bb9ba3a3f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:50.560930 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79895d0-5a51-4fc3-b022-599bb9ba3a3f None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:50.582367 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 369/736] 10.222.0.61 () {60 vars in 1141 bytes} [Fri Aug 26 12:01:50 2022] POST /identity/v3/services => generated 240 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:01:50.611819 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5e2e4a-11cc-4216-8d9d-bea9d748d793 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:50.634534 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:50.676740 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54648ca7-0e4b-48d2-945c-a4825aa33434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:50.677764 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54648ca7-0e4b-48d2-945c-a4825aa33434 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:50.678243 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54648ca7-0e4b-48d2-945c-a4825aa33434 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:50.678725 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54648ca7-0e4b-48d2-945c-a4825aa33434 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:50.692096 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79895d0-5a51-4fc3-b022-599bb9ba3a3f None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:50.694940 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54648ca7-0e4b-48d2-945c-a4825aa33434 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:50.696664 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54648ca7-0e4b-48d2-945c-a4825aa33434 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:50.708235 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:50.713013 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-54648ca7-0e4b-48d2-945c-a4825aa33434 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:50.713013 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:50.713013 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:50.713013 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:50.713013 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:50.713013 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:50.713013 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:50.713013 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:50.713013 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:50.713013 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:50.713013 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:50.713013 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:50.713013 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:01:50.713013 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:01:50.713013 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:01:50.713013 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:01:50.714042 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:50.714042 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:50.714042 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:50.714042 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:50.714042 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:50.714042 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:50.714042 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:50.714042 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:50.714042 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:50.714042 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:50.714042 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:50.714042 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:50.714042 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:50.714042 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:50.714042 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:50.714042 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:50.714042 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:01:50.714950 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:01:50.714950 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:50.714950 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:50.714950 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 368/737] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:01:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:50.721558 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54648ca7-0e4b-48d2-945c-a4825aa33434 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:50.830891 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79895d0-5a51-4fc3-b022-599bb9ba3a3f None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:50.869751 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00f70a5-1792-4bc3-9085-e2ff44edfeee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:50.870483 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00f70a5-1792-4bc3-9085-e2ff44edfeee None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:50.870868 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00f70a5-1792-4bc3-9085-e2ff44edfeee None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:50.871233 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00f70a5-1792-4bc3-9085-e2ff44edfeee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:50.873775 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54648ca7-0e4b-48d2-945c-a4825aa33434 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:50.882566 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00f70a5-1792-4bc3-9085-e2ff44edfeee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:01:50.888532 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-c00f70a5-1792-4bc3-9085-e2ff44edfeee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 12:01:50.888532 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:50.888532 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:50.888532 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:50.888532 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:50.888532 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:50.888532 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:50.888532 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:50.888532 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:50.888532 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:50.888532 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:50.888532 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:50.888532 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:01:50.888532 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:01:50.888532 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:01:50.888532 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:01:50.889567 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:01:50.889567 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:01:50.889567 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:01:50.889567 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:01:50.889567 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:50.889567 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:50.889567 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:50.889567 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:50.889567 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:50.889567 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:50.889567 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:50.889567 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:50.889567 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:50.889567 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:50.889567 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:50.889567 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:50.890411 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:50.890411 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:50.890411 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:50.890411 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:50.890411 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:01:50.890411 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:01:50.890411 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:01:50.890411 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:01:50.890411 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:01:50.890411 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:01:50.890411 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 12:01:50.890411 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:50.890411 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 370/738] 10.222.0.61 () {56 vars in 1096 bytes} [Fri Aug 26 12:01:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:50.896111 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00f70a5-1792-4bc3-9085-e2ff44edfeee None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:50.928209 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a30182-34a4-4c26-af8b-5274fa8a4f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:50.935255 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a30182-34a4-4c26-af8b-5274fa8a4f53 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:50.935255 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a30182-34a4-4c26-af8b-5274fa8a4f53 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:50.935255 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a30182-34a4-4c26-af8b-5274fa8a4f53 None admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:50.935255 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a30182-34a4-4c26-af8b-5274fa8a4f53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:50.935255 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a30182-34a4-4c26-af8b-5274fa8a4f53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:50.935255 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a30182-34a4-4c26-af8b-5274fa8a4f53 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:50.962125 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 369/739] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:01:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:50.971363 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a30182-34a4-4c26-af8b-5274fa8a4f53 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:51.018864 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:51.021119 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:51.098563 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a30182-34a4-4c26-af8b-5274fa8a4f53 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:51.117479 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00f70a5-1792-4bc3-9085-e2ff44edfeee None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:51.135862 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b717ab-ada2-4f51-8f2a-084cfa903b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:51.136497 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b717ab-ada2-4f51-8f2a-084cfa903b34 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:51.136779 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b717ab-ada2-4f51-8f2a-084cfa903b34 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:51.137056 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b717ab-ada2-4f51-8f2a-084cfa903b34 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:51.144789 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b717ab-ada2-4f51-8f2a-084cfa903b34 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:51.146281 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b717ab-ada2-4f51-8f2a-084cfa903b34 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:51.158314 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-28b717ab-ada2-4f51-8f2a-084cfa903b34 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:01:51.158314 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:51.158314 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:51.158314 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:51.158314 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:51.158314 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:51.158314 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:51.158314 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:51.158314 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:51.158314 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:51.158314 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:51.158314 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:51.158314 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 12:01:51.158314 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 12:01:51.158314 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 12:01:51.159289 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 12:01:51.159289 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:51.159289 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:51.159289 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:51.159289 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:51.159289 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:51.159289 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:51.159289 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:51.159289 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:51.159289 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:51.159289 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:51.159289 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:51.159289 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:51.159289 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:51.159289 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:51.159289 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:51.159289 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:51.160216 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 12:01:51.160216 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 12:01:51.160216 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 12:01:51.160216 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 12:01:51.160216 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 12:01:51.160216 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 12:01:51.160216 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:01:51.160216 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:51.160216 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 370/740] 10.222.0.61 () {56 vars in 1111 bytes} [Fri Aug 26 12:01:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:51.165574 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b717ab-ada2-4f51-8f2a-084cfa903b34 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:51.196849 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217003ee-1716-4103-bbb6-e6d5eb1b6a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:51.197801 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217003ee-1716-4103-bbb6-e6d5eb1b6a0b None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:51.198284 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217003ee-1716-4103-bbb6-e6d5eb1b6a0b None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:51.198754 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217003ee-1716-4103-bbb6-e6d5eb1b6a0b None admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:51.199671 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217003ee-1716-4103-bbb6-e6d5eb1b6a0b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:51.200247 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217003ee-1716-4103-bbb6-e6d5eb1b6a0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:51.201826 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217003ee-1716-4103-bbb6-e6d5eb1b6a0b None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:51.220849 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 371/741] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:01:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:01:51.225941 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217003ee-1716-4103-bbb6-e6d5eb1b6a0b None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:51.279350 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b717ab-ada2-4f51-8f2a-084cfa903b34 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:51.285390 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:51.318999 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351f9e31-ed38-41d8-8f97-db75f8af4461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:51.319708 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f9e31-ed38-41d8-8f97-db75f8af4461 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:51.319991 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f9e31-ed38-41d8-8f97-db75f8af4461 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:51.320287 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f9e31-ed38-41d8-8f97-db75f8af4461 None admin] PATH_INFO: `/v3/projects` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:51.321125 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351f9e31-ed38-41d8-8f97-db75f8af4461 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:51.321656 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351f9e31-ed38-41d8-8f97-db75f8af4461 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:51.323698 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351f9e31-ed38-41d8-8f97-db75f8af4461 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:51.342314 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 371/742] 10.222.0.61 () {56 vars in 1148 bytes} [Fri Aug 26 12:01:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:51.344532 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd9391-2dcb-465e-9d61-a1c3c7abfca4 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:51.344821 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd9391-2dcb-465e-9d61-a1c3c7abfca4 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:51.345101 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcd9391-2dcb-465e-9d61-a1c3c7abfca4 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:51.351327 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 372/743] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:01:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:01:51.352110 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcd9391-2dcb-465e-9d61-a1c3c7abfca4 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:51.388368 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217003ee-1716-4103-bbb6-e6d5eb1b6a0b None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:51.417617 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:51.428167 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0572b7-2ac0-4887-9ab6-2e07ce1da85b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:51.428847 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0572b7-2ac0-4887-9ab6-2e07ce1da85b None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:51.429128 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0572b7-2ac0-4887-9ab6-2e07ce1da85b None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:51.429395 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0572b7-2ac0-4887-9ab6-2e07ce1da85b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:51.442094 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0572b7-2ac0-4887-9ab6-2e07ce1da85b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:51.443660 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0572b7-2ac0-4887-9ab6-2e07ce1da85b None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:51.458869 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-3a0572b7-2ac0-4887-9ab6-2e07ce1da85b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:51.458869 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:51.458869 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:51.458869 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:51.458869 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:51.458869 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:51.458869 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:51.458869 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:51.458869 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:51.458869 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:51.458869 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:51.458869 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:51.458869 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:01:51.458869 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:01:51.458869 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:01:51.458869 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:01:51.460112 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:51.460112 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:51.460112 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:51.460112 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:51.460112 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:51.460112 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:51.460112 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:51.460112 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:51.460112 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:51.460112 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:51.460112 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:51.460112 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:51.460112 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:51.460112 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:51.460112 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:51.460112 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:51.460112 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:01:51.461199 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:01:51.461199 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:51.461199 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:51.461199 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 372/744] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:01:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 12:01:51.461642 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2dbac0-7cef-47fc-8912-e4d217a6e321 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:51.462171 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2dbac0-7cef-47fc-8912-e4d217a6e321 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:51.462272 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2dbac0-7cef-47fc-8912-e4d217a6e321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:51.484888 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-1f2dbac0-7cef-47fc-8912-e4d217a6e321 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:01:51.486247 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-1f2dbac0-7cef-47fc-8912-e4d217a6e321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:01:51.490671 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcd9391-2dcb-465e-9d61-a1c3c7abfca4 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:51.537616 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ec43c-ed7f-45db-bfdb-175a71efb184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:51.538701 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ec43c-ed7f-45db-bfdb-175a71efb184 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:51.539133 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ec43c-ed7f-45db-bfdb-175a71efb184 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:51.540041 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ec43c-ed7f-45db-bfdb-175a71efb184 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:51.541198 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17ec43c-ed7f-45db-bfdb-175a71efb184 None admin] RBAC: Adding query filter params (role.id=70ec1ba484714c8ca3adb8fe5845107c, scope.project.id=ecd68423a83a427681d44dbf266fce6e, user.id=ead95a244d11432d9fe9f718e1a75f38) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:51.541655 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17ec43c-ed7f-45db-bfdb-175a71efb184 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:51.544527 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17ec43c-ed7f-45db-bfdb-175a71efb184 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:51.556935 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 373/745] 10.222.0.61 () {56 vars in 1374 bytes} [Fri Aug 26 12:01:51 2022] GET /identity/v3/role_assignments?user.id=ead95a244d11432d9fe9f718e1a75f38&scope.project.id=ecd68423a83a427681d44dbf266fce6e&role.id=70ec1ba484714c8ca3adb8fe5845107c => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:51.556935 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ec43c-ed7f-45db-bfdb-175a71efb184 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:51.583432 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 373/746] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:01:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 12:01:51.592125 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:51.725542 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ec43c-ed7f-45db-bfdb-175a71efb184 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:51.772928 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e631db-4b7d-4845-9cbc-0ec385fbb0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:51.773553 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e631db-4b7d-4845-9cbc-0ec385fbb0c1 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:51.773834 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e631db-4b7d-4845-9cbc-0ec385fbb0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:51.774110 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e631db-4b7d-4845-9cbc-0ec385fbb0c1 None admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:51.774761 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e631db-4b7d-4845-9cbc-0ec385fbb0c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:51.775064 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e631db-4b7d-4845-9cbc-0ec385fbb0c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:51.776479 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e631db-4b7d-4845-9cbc-0ec385fbb0c1 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:51.792576 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 374/747] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:01:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:51.799807 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2dbac0-7cef-47fc-8912-e4d217a6e321 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:52.039928 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2dbac0-7cef-47fc-8912-e4d217a6e321 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:52.069086 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:52.109413 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cd8bc0-43cb-4add-954f-90bd51c06032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:52.110045 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd8bc0-43cb-4add-954f-90bd51c06032 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:52.110325 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd8bc0-43cb-4add-954f-90bd51c06032 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:52.110591 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd8bc0-43cb-4add-954f-90bd51c06032 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:52.123680 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cd8bc0-43cb-4add-954f-90bd51c06032 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:52.125130 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cd8bc0-43cb-4add-954f-90bd51c06032 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:52.141659 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-c0cd8bc0-43cb-4add-954f-90bd51c06032 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 12:01:52.141659 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:52.141659 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:52.141659 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:52.141659 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:52.141659 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:52.141659 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:52.141659 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:52.141659 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:52.141659 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:52.141659 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:52.141659 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:52.141659 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 12:01:52.141659 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 12:01:52.141659 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 12:01:52.141659 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 12:01:52.143621 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:52.143621 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:52.143621 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 12:01:52.143621 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:01:52.143621 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 12:01:52.143621 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:01:52.143621 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:52.143621 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:52.143621 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:52.143621 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:52.143621 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:52.143621 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:52.143621 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:52.143621 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:52.143621 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:52.143621 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:52.143621 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:52.143621 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:52.145694 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:52.145694 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:52.145694 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 12:01:52.145694 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 12:01:52.145694 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 12:01:52.145694 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 12:01:52.145694 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 12:01:52.145694 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 12:01:52.145694 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 12:01:52.145694 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:52.145694 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 374/748] 10.222.0.61 () {56 vars in 1093 bytes} [Fri Aug 26 12:01:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:52.151161 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e631db-4b7d-4845-9cbc-0ec385fbb0c1 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:52.209468 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cd8bc0-43cb-4add-954f-90bd51c06032 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:52.305646 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:52.343406 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e631db-4b7d-4845-9cbc-0ec385fbb0c1 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:52.380630 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1d2420-2326-43ae-9376-4b237c7f23df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:52.381237 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d2420-2326-43ae-9376-4b237c7f23df None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:52.381532 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d2420-2326-43ae-9376-4b237c7f23df None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:52.381803 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d2420-2326-43ae-9376-4b237c7f23df None admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:52.382516 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1d2420-2326-43ae-9376-4b237c7f23df None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:52.382826 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1d2420-2326-43ae-9376-4b237c7f23df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:52.384696 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1d2420-2326-43ae-9376-4b237c7f23df None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:52.404635 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 375/749] 10.222.0.61 () {56 vars in 1133 bytes} [Fri Aug 26 12:01:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:01:52.409950 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1d2420-2326-43ae-9376-4b237c7f23df None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:52.518777 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1d2420-2326-43ae-9376-4b237c7f23df None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:52.542498 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cd8bc0-43cb-4add-954f-90bd51c06032 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:52.551817 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:52.557924 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dcf2ea-c3e3-406a-b873-692f275bb9fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:52.558558 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dcf2ea-c3e3-406a-b873-692f275bb9fd None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:52.558838 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dcf2ea-c3e3-406a-b873-692f275bb9fd None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:52.559110 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dcf2ea-c3e3-406a-b873-692f275bb9fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:52.570815 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dcf2ea-c3e3-406a-b873-692f275bb9fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:52.572437 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dcf2ea-c3e3-406a-b873-692f275bb9fd None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:52.589347 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-e3dcf2ea-c3e3-406a-b873-692f275bb9fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:52.589347 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:52.589347 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:52.589347 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:52.589347 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:52.589347 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:52.589347 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:52.589347 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:52.589347 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:52.589347 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:52.589347 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:52.589347 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:52.589347 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:01:52.589347 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:01:52.589347 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:01:52.589347 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:01:52.591695 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:52.591695 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:52.591695 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:52.591695 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:52.591695 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:52.591695 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:52.591695 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:52.591695 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:52.591695 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:52.591695 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:52.591695 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:52.591695 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:52.591695 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:52.591695 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:52.591695 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:52.591695 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:52.591695 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:01:52.594307 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:01:52.594307 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:52.594307 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:52.594307 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 376/750] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:01:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 12:01:52.597567 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dcf2ea-c3e3-406a-b873-692f275bb9fd None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:52.708887 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dcf2ea-c3e3-406a-b873-692f275bb9fd None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:52.728315 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640de6e0-0508-4485-a245-bd99138ee4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:52.729328 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640de6e0-0508-4485-a245-bd99138ee4a5 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:52.729819 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640de6e0-0508-4485-a245-bd99138ee4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:52.730283 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640de6e0-0508-4485-a245-bd99138ee4a5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:52.745733 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be98652f-3172-4067-9d52-0ecd259e4632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:52.746353 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98652f-3172-4067-9d52-0ecd259e4632 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:52.746645 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98652f-3172-4067-9d52-0ecd259e4632 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:52.747419 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98652f-3172-4067-9d52-0ecd259e4632 None admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:52.747989 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be98652f-3172-4067-9d52-0ecd259e4632 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:52.748343 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be98652f-3172-4067-9d52-0ecd259e4632 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:52.749735 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be98652f-3172-4067-9d52-0ecd259e4632 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:52.763790 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 377/751] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:01:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:01:52.769185 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be98652f-3172-4067-9d52-0ecd259e4632 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:52.788141 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640de6e0-0508-4485-a245-bd99138ee4a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:01:52.806319 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-640de6e0-0508-4485-a245-bd99138ee4a5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 12:01:52.806319 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:52.806319 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:52.806319 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:52.806319 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:52.806319 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:52.806319 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:52.806319 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:52.806319 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:52.806319 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:52.806319 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:52.806319 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:52.806319 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:01:52.806319 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:01:52.806319 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:01:52.806319 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:01:52.808138 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:01:52.808138 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:01:52.808138 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:01:52.808138 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:01:52.808138 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:52.808138 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:52.808138 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:52.808138 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:52.808138 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:52.808138 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:52.808138 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:52.808138 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:52.808138 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:52.808138 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:52.808138 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:52.808138 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:52.809822 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:52.809822 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:52.809822 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:52.809822 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:52.809822 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:01:52.809822 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:01:52.809822 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:01:52.809822 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:01:52.809822 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:01:52.809822 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:01:52.809822 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 12:01:52.809822 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:52.813421 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 375/752] 10.222.0.61 () {56 vars in 1102 bytes} [Fri Aug 26 12:01:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 599 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:52.824830 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640de6e0-0508-4485-a245-bd99138ee4a5 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:52.827575 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:52.890737 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be98652f-3172-4067-9d52-0ecd259e4632 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:52.928290 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f860a8-e734-43c1-82e3-605683db9135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:52.928927 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f860a8-e734-43c1-82e3-605683db9135 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:52.929369 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f860a8-e734-43c1-82e3-605683db9135 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:52.929650 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f860a8-e734-43c1-82e3-605683db9135 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:52.938205 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f860a8-e734-43c1-82e3-605683db9135 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:52.939712 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f860a8-e734-43c1-82e3-605683db9135 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:52.954259 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-27f860a8-e734-43c1-82e3-605683db9135 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:01:52.954259 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:52.954259 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:52.954259 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:52.954259 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:52.954259 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:52.954259 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:52.954259 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:52.954259 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:52.954259 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:52.954259 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:52.954259 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:52.954259 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 12:01:52.954259 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 12:01:52.954259 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 12:01:52.957711 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 12:01:52.957711 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:52.957711 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:52.957711 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:52.957711 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:52.957711 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:52.957711 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:52.957711 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:52.957711 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:52.957711 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:52.957711 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:52.957711 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:52.957711 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:52.957711 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:52.957711 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:52.957711 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:52.957711 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:52.961241 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 12:01:52.961241 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 12:01:52.961241 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 12:01:52.961241 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 12:01:52.961241 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 12:01:52.961241 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 12:01:52.961241 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:01:52.961241 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:52.961241 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 378/753] 10.222.0.61 () {56 vars in 1111 bytes} [Fri Aug 26 12:01:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:52.971341 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f860a8-e734-43c1-82e3-605683db9135 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:52.976035 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:53.076426 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f860a8-e734-43c1-82e3-605683db9135 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:53.081490 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:53.119691 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191bb4b5-003f-4a6e-a466-bc1cfe9ac3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:53.120406 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191bb4b5-003f-4a6e-a466-bc1cfe9ac3b7 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:53.120905 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191bb4b5-003f-4a6e-a466-bc1cfe9ac3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:53.121182 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191bb4b5-003f-4a6e-a466-bc1cfe9ac3b7 None admin] PATH_INFO: `/v3/projects` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:53.122560 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191bb4b5-003f-4a6e-a466-bc1cfe9ac3b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:53.123300 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191bb4b5-003f-4a6e-a466-bc1cfe9ac3b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:53.124751 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191bb4b5-003f-4a6e-a466-bc1cfe9ac3b7 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:53.140387 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 379/754] 10.222.0.61 () {56 vars in 1148 bytes} [Fri Aug 26 12:01:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:53.144108 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062a1326-e5b1-4a4e-bee2-3b198148b208 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:53.144543 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062a1326-e5b1-4a4e-bee2-3b198148b208 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:53.145024 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062a1326-e5b1-4a4e-bee2-3b198148b208 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:53.147788 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 380/755] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:01:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:01:53.149182 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062a1326-e5b1-4a4e-bee2-3b198148b208 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:53.160179 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640de6e0-0508-4485-a245-bd99138ee4a5 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:53.200711 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41095aea-ff3f-4091-9493-4931cfd290f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:53.201822 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41095aea-ff3f-4091-9493-4931cfd290f0 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:53.202106 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41095aea-ff3f-4091-9493-4931cfd290f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:53.202381 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41095aea-ff3f-4091-9493-4931cfd290f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:53.203050 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41095aea-ff3f-4091-9493-4931cfd290f0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:53.203382 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41095aea-ff3f-4091-9493-4931cfd290f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:53.205512 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41095aea-ff3f-4091-9493-4931cfd290f0 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:53.215604 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 376/756] 10.222.0.61 () {56 vars in 1103 bytes} [Fri Aug 26 12:01:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 392 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:53.217231 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8944923-245c-4d32-8d6b-791000b9a524 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:53.217526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8944923-245c-4d32-8d6b-791000b9a524 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:53.217803 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8944923-245c-4d32-8d6b-791000b9a524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:53.244477 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-f8944923-245c-4d32-8d6b-791000b9a524 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:01:53.245729 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-f8944923-245c-4d32-8d6b-791000b9a524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:01:53.326591 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062a1326-e5b1-4a4e-bee2-3b198148b208 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:53.344763 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 377/757] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:01:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 12:01:53.346530 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8944923-245c-4d32-8d6b-791000b9a524 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:53.369559 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6da3938-21b3-410c-bfea-144f6c9a8535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:53.370228 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da3938-21b3-410c-bfea-144f6c9a8535 None admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:53.370510 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da3938-21b3-410c-bfea-144f6c9a8535 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:53.370790 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6da3938-21b3-410c-bfea-144f6c9a8535 None admin] PATH_INFO: `/v3/projects/ecd68423a83a427681d44dbf266fce6e/users/7aacd7d6d17e4f10ba5c9107a12beafa/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:53.373914 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6da3938-21b3-410c-bfea-144f6c9a8535 None admin] RBAC: Authorizing `identity:create_grant(project_id=ecd68423a83a427681d44dbf266fce6e, user_id=7aacd7d6d17e4f10ba5c9107a12beafa, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:53.375456 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6da3938-21b3-410c-bfea-144f6c9a8535 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:53.392447 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 381/758] 10.222.0.61 () {58 vars in 1439 bytes} [Fri Aug 26 12:01:53 2022] PUT /identity/v3/projects/ecd68423a83a427681d44dbf266fce6e/users/7aacd7d6d17e4f10ba5c9107a12beafa/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:01:53.426849 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:53.472178 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef7f2a8-8c4b-40c7-b76b-21435cda166c None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:53.472178 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef7f2a8-8c4b-40c7-b76b-21435cda166c None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:53.474767 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef7f2a8-8c4b-40c7-b76b-21435cda166c None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:53.475737 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 382/759] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:01:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 12:01:53.482603 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e09ed1-acfd-4f16-b447-e6ff2d2024a2 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:53.483248 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e09ed1-acfd-4f16-b447-e6ff2d2024a2 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:53.483758 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e09ed1-acfd-4f16-b447-e6ff2d2024a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:53.504224 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-f6e09ed1-acfd-4f16-b447-e6ff2d2024a2 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:01:53.505764 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-f6e09ed1-acfd-4f16-b447-e6ff2d2024a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:01:53.527774 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:53.536059 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8944923-245c-4d32-8d6b-791000b9a524 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:53.599859 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31461ea-ced4-413e-821b-dac06dfffc2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:53.601106 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31461ea-ced4-413e-821b-dac06dfffc2f None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:53.603055 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31461ea-ced4-413e-821b-dac06dfffc2f None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:53.603564 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31461ea-ced4-413e-821b-dac06dfffc2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:53.620053 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31461ea-ced4-413e-821b-dac06dfffc2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:53.621627 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31461ea-ced4-413e-821b-dac06dfffc2f None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:53.640926 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-a31461ea-ced4-413e-821b-dac06dfffc2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:53.640926 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:53.640926 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:53.640926 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:53.640926 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:53.640926 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:53.640926 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:53.640926 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:53.640926 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:53.640926 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:53.640926 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:53.640926 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:53.640926 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:01:53.640926 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:01:53.640926 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:01:53.640926 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:01:53.641937 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:53.641937 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:53.641937 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:53.641937 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:53.641937 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:53.641937 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:53.641937 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:53.641937 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:53.641937 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:53.641937 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:53.641937 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:53.641937 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:53.641937 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:53.641937 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:53.641937 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:53.641937 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:53.641937 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:01:53.642836 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:01:53.642836 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:53.642836 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:53.645527 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 378/760] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:01:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:53.648618 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31461ea-ced4-413e-821b-dac06dfffc2f None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:53.668453 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 383/761] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:01:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 12:01:53.710100 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e09ed1-acfd-4f16-b447-e6ff2d2024a2 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:53.780212 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:53.812084 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31461ea-ced4-413e-821b-dac06dfffc2f None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:53.848687 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc30889-fc61-4add-aaea-6cf63087457e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:53.849570 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc30889-fc61-4add-aaea-6cf63087457e None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:53.849989 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc30889-fc61-4add-aaea-6cf63087457e None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:53.850402 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc30889-fc61-4add-aaea-6cf63087457e None admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:53.851262 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc30889-fc61-4add-aaea-6cf63087457e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:53.851751 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc30889-fc61-4add-aaea-6cf63087457e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:53.853288 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc30889-fc61-4add-aaea-6cf63087457e None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:53.866592 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 379/762] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:01:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:01:53.872746 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc30889-fc61-4add-aaea-6cf63087457e None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:53.904061 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e09ed1-acfd-4f16-b447-e6ff2d2024a2 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:53.968760 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af652b4-9d84-4be2-a158-cb4ac104f14a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:53.969385 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af652b4-9d84-4be2-a158-cb4ac104f14a None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:53.969679 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af652b4-9d84-4be2-a158-cb4ac104f14a None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:53.969956 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af652b4-9d84-4be2-a158-cb4ac104f14a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:53.970613 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af652b4-9d84-4be2-a158-cb4ac104f14a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:53.972025 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af652b4-9d84-4be2-a158-cb4ac104f14a None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:53.980948 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-2af652b4-9d84-4be2-a158-cb4ac104f14a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 12:01:53.980948 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:53.980948 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 12:01:53.980948 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 12:01:53.980948 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 12:01:53.980948 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 12:01:53.980948 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 12:01:53.980948 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 12:01:53.980948 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 12:01:53.980948 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:53.980948 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 12:01:53.980948 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:53.980948 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:53.980948 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:53.980948 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:53.980948 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:53.980948 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:53.982553 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:53.982553 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:53.982553 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:53.982553 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:53.982553 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:53.982553 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:53.982553 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 12:01:53.982553 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 12:01:53.982553 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 12:01:53.982553 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 12:01:53.982553 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:53.982553 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:53.982553 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:53.982553 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:53.982553 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:53.982553 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:53.984134 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:53.984134 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:53.984134 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:53.984134 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:53.984134 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:53.984134 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:53.984134 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:53.984134 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:53.984134 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:53.984134 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:53.984134 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 12:01:53.984134 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 12:01:53.984134 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 12:01:53.984134 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:53.984134 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 384/763] 10.222.0.61 () {56 vars in 1129 bytes} [Fri Aug 26 12:01:53 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 12:01:53.984925 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af652b4-9d84-4be2-a158-cb4ac104f14a None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:54.005387 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:54.025487 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc30889-fc61-4add-aaea-6cf63087457e None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:54.069624 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ed1ca3-6dff-48e2-8da0-b7b991297a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:54.070363 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ed1ca3-6dff-48e2-8da0-b7b991297a6c None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:54.070701 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ed1ca3-6dff-48e2-8da0-b7b991297a6c None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:54.071062 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ed1ca3-6dff-48e2-8da0-b7b991297a6c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:54.073062 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:54.081026 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ed1ca3-6dff-48e2-8da0-b7b991297a6c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:54.082475 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ed1ca3-6dff-48e2-8da0-b7b991297a6c None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:54.097071 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-e4ed1ca3-6dff-48e2-8da0-b7b991297a6c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 12:01:54.097071 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:54.097071 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:54.097071 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:54.097071 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:54.097071 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:54.097071 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:54.097071 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:54.097071 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:54.097071 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:54.097071 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:54.097071 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:54.097071 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 12:01:54.097071 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 12:01:54.097071 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 12:01:54.097071 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 12:01:54.098052 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:54.098052 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:54.098052 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 12:01:54.098052 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:01:54.098052 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 12:01:54.098052 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:01:54.098052 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:54.098052 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:54.098052 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:54.098052 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:54.098052 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:54.098052 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:54.098052 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:54.098052 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:54.098052 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:54.098052 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:54.098052 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:54.098052 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:54.099018 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:54.099018 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:54.099018 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 12:01:54.099018 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 12:01:54.099018 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 12:01:54.099018 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 12:01:54.099018 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 12:01:54.099018 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 12:01:54.099018 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 12:01:54.099018 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:54.099018 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 380/764] 10.222.0.61 () {56 vars in 1117 bytes} [Fri Aug 26 12:01:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 12:01:54.099650 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ed1ca3-6dff-48e2-8da0-b7b991297a6c None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:54.104038 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af652b4-9d84-4be2-a158-cb4ac104f14a None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:54.141841 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25400d2f-3c9d-4a7f-96b0-64f6d143b983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:54.142532 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25400d2f-3c9d-4a7f-96b0-64f6d143b983 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:54.142817 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25400d2f-3c9d-4a7f-96b0-64f6d143b983 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:54.143087 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25400d2f-3c9d-4a7f-96b0-64f6d143b983 None admin] PATH_INFO: `/v3/services/network` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:54.143783 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25400d2f-3c9d-4a7f-96b0-64f6d143b983 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:54.145245 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25400d2f-3c9d-4a7f-96b0-64f6d143b983 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:54.154801 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-25400d2f-3c9d-4a7f-96b0-64f6d143b983 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 12:01:54.154801 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:54.154801 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 12:01:54.154801 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 12:01:54.154801 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 12:01:54.154801 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 12:01:54.154801 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 12:01:54.154801 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 12:01:54.154801 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 12:01:54.154801 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:54.154801 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 12:01:54.154801 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:54.154801 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:54.154801 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:54.154801 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:54.154801 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:54.154801 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:54.157086 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:54.157086 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:54.157086 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:54.157086 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:54.157086 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:54.157086 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:54.157086 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 12:01:54.157086 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 12:01:54.157086 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 12:01:54.157086 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 12:01:54.157086 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:54.157086 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:54.157086 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:54.157086 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:54.157086 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:54.157086 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:54.159042 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:54.159042 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:54.159042 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:54.159042 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:54.159042 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:54.159042 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:54.159042 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:54.159042 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:54.159042 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:54.159042 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:54.159042 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 12:01:54.159042 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 12:01:54.159042 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 12:01:54.159042 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:54.159042 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 385/765] 10.222.0.61 () {56 vars in 1111 bytes} [Fri Aug 26 12:01:53 2022] GET /identity/v3/services/network => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:54.160894 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25400d2f-3c9d-4a7f-96b0-64f6d143b983 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:54.216044 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ed1ca3-6dff-48e2-8da0-b7b991297a6c None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:54.242245 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:54.256044 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15688228-b5a1-4001-8521-bcfc7eb52699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:54.257141 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15688228-b5a1-4001-8521-bcfc7eb52699 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:54.257606 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15688228-b5a1-4001-8521-bcfc7eb52699 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:54.257884 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15688228-b5a1-4001-8521-bcfc7eb52699 None admin] PATH_INFO: `/v3/services` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:54.258787 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15688228-b5a1-4001-8521-bcfc7eb52699 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:54.259091 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15688228-b5a1-4001-8521-bcfc7eb52699 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:54.260503 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15688228-b5a1-4001-8521-bcfc7eb52699 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:54.265042 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 381/766] 10.222.0.61 () {56 vars in 1124 bytes} [Fri Aug 26 12:01:54 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:54.265819 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15688228-b5a1-4001-8521-bcfc7eb52699 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:54.266229 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25400d2f-3c9d-4a7f-96b0-64f6d143b983 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:54.305397 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02468717-0f64-44d5-a6f4-d942e3c0689b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:54.306000 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02468717-0f64-44d5-a6f4-d942e3c0689b None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:54.306306 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02468717-0f64-44d5-a6f4-d942e3c0689b None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:54.306545 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02468717-0f64-44d5-a6f4-d942e3c0689b None admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:54.307442 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02468717-0f64-44d5-a6f4-d942e3c0689b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:54.307745 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02468717-0f64-44d5-a6f4-d942e3c0689b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:54.309318 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02468717-0f64-44d5-a6f4-d942e3c0689b None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:54.323558 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:54.333805 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 386/767] 10.222.0.61 () {56 vars in 1149 bytes} [Fri Aug 26 12:01:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:54.335091 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35bbaf8-8609-48c3-bb8c-811336bb4c0c None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:54.335408 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35bbaf8-8609-48c3-bb8c-811336bb4c0c None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:54.335689 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35bbaf8-8609-48c3-bb8c-811336bb4c0c None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:54.337585 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 387/768] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:01:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:01:54.337842 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35bbaf8-8609-48c3-bb8c-811336bb4c0c None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:54.370700 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15688228-b5a1-4001-8521-bcfc7eb52699 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:54.406482 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b11b1-055c-4422-a4a6-e35f38bda9e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:54.407083 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b11b1-055c-4422-a4a6-e35f38bda9e7 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:54.407445 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b11b1-055c-4422-a4a6-e35f38bda9e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:54.407725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b11b1-055c-4422-a4a6-e35f38bda9e7 None admin] PATH_INFO: `/v3/services` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:54.408375 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1b11b1-055c-4422-a4a6-e35f38bda9e7 None admin] RBAC: Adding query filter params (name=network) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:54.408680 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1b11b1-055c-4422-a4a6-e35f38bda9e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:54.410048 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1b11b1-055c-4422-a4a6-e35f38bda9e7 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:54.419266 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 382/769] 10.222.0.61 () {56 vars in 1112 bytes} [Fri Aug 26 12:01:54 2022] GET /identity/v3/services?name=network => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:54.419266 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b11b1-055c-4422-a4a6-e35f38bda9e7 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:54.451577 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35bbaf8-8609-48c3-bb8c-811336bb4c0c None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:54.485921 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f175c476-39c6-4d65-b968-7d263fb50257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:54.486520 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f175c476-39c6-4d65-b968-7d263fb50257 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:54.486796 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f175c476-39c6-4d65-b968-7d263fb50257 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:54.487066 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f175c476-39c6-4d65-b968-7d263fb50257 None admin] PATH_INFO: `/v3/services` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:54.487723 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f175c476-39c6-4d65-b968-7d263fb50257 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:54.488021 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f175c476-39c6-4d65-b968-7d263fb50257 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:54.489369 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f175c476-39c6-4d65-b968-7d263fb50257 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:54.491403 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:54.499728 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 388/770] 10.222.0.61 () {56 vars in 1124 bytes} [Fri Aug 26 12:01:54 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:54.500858 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39786206-7453-4bdc-9598-9026cc00f915 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:54.501145 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39786206-7453-4bdc-9598-9026cc00f915 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:54.501417 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39786206-7453-4bdc-9598-9026cc00f915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:54.519065 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-39786206-7453-4bdc-9598-9026cc00f915 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:01:54.520255 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-39786206-7453-4bdc-9598-9026cc00f915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:01:54.527154 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b11b1-055c-4422-a4a6-e35f38bda9e7 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:54.556919 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:54.569896 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6248baf-2495-49d6-8c39-21f9c006f957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:54.570588 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6248baf-2495-49d6-8c39-21f9c006f957 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:54.570935 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6248baf-2495-49d6-8c39-21f9c006f957 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:54.571228 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6248baf-2495-49d6-8c39-21f9c006f957 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:54.593419 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 389/771] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:01:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 12:01:54.594427 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39786206-7453-4bdc-9598-9026cc00f915 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:54.655709 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6248baf-2495-49d6-8c39-21f9c006f957 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:54.657534 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6248baf-2495-49d6-8c39-21f9c006f957 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:54.725008 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39786206-7453-4bdc-9598-9026cc00f915 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:54.773714 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 383/772] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:01:54 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 12:01:54.774976 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6248baf-2495-49d6-8c39-21f9c006f957 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:54.782392 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:54.790158 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f00109f-c1d4-4d8d-9333-9cd826e479d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:54.791131 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f00109f-c1d4-4d8d-9333-9cd826e479d9 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:54.791657 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f00109f-c1d4-4d8d-9333-9cd826e479d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:54.792126 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f00109f-c1d4-4d8d-9333-9cd826e479d9 None admin] PATH_INFO: `/v3/services` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:54.793052 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f00109f-c1d4-4d8d-9333-9cd826e479d9 None admin] RBAC: Adding query filter params (type=network) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:54.793577 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f00109f-c1d4-4d8d-9333-9cd826e479d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:54.795981 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f00109f-c1d4-4d8d-9333-9cd826e479d9 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:54.801820 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 390/773] 10.222.0.61 () {56 vars in 1112 bytes} [Fri Aug 26 12:01:54 2022] GET /identity/v3/services?type=network => generated 350 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:54.802907 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f00109f-c1d4-4d8d-9333-9cd826e479d9 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:54.809328 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:54.905828 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6248baf-2495-49d6-8c39-21f9c006f957 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:54.915022 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f00109f-c1d4-4d8d-9333-9cd826e479d9 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:54.947818 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ab26c5-a6af-4e49-873d-d40f1c4099c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:54.948426 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ab26c5-a6af-4e49-873d-d40f1c4099c8 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:54.948719 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ab26c5-a6af-4e49-873d-d40f1c4099c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:54.948982 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ab26c5-a6af-4e49-873d-d40f1c4099c8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:54.949675 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ab26c5-a6af-4e49-873d-d40f1c4099c8 None admin] RBAC: Adding query filter params (interface=public, service_id=76821146c90b411fbea86b786cb3fb83, region_id=RegionOne) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:54.950062 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ab26c5-a6af-4e49-873d-d40f1c4099c8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:54.950666 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d884130e-ce2e-449b-a032-49578a257b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:54.951324 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d884130e-ce2e-449b-a032-49578a257b79 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:54.951422 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ab26c5-a6af-4e49-873d-d40f1c4099c8 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:54.951604 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d884130e-ce2e-449b-a032-49578a257b79 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:54.951875 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d884130e-ce2e-449b-a032-49578a257b79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:54.958870 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 384/774] 10.222.0.61 () {56 vars in 1251 bytes} [Fri Aug 26 12:01:54 2022] GET /identity/v3/endpoints?service_id=76821146c90b411fbea86b786cb3fb83&interface=public®ion_id=RegionOne => generated 194 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:54.962592 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d884130e-ce2e-449b-a032-49578a257b79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:54.963975 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ab26c5-a6af-4e49-873d-d40f1c4099c8 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:54.964174 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d884130e-ce2e-449b-a032-49578a257b79 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:54.979294 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-d884130e-ce2e-449b-a032-49578a257b79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:54.979294 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:54.979294 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:54.979294 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:54.979294 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:54.979294 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:54.979294 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:54.979294 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:54.979294 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:54.979294 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:54.979294 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:54.979294 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:54.979294 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:01:54.979294 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:01:54.979294 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:01:54.979294 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:01:54.980302 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:54.980302 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:54.980302 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:54.980302 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:54.980302 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:54.980302 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:54.980302 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:54.980302 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:54.980302 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:54.980302 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:54.980302 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:54.980302 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:54.980302 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:54.980302 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:54.980302 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:54.980302 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:54.980302 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:01:54.981735 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:01:54.981735 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:54.981735 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:54.985444 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 391/775] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:01:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:54.986355 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d884130e-ce2e-449b-a032-49578a257b79 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:55.035920 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:55.057931 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:55.076514 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ab26c5-a6af-4e49-873d-d40f1c4099c8 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:55.094999 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d884130e-ce2e-449b-a032-49578a257b79 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:55.113172 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e5d494-5db7-44a3-a886-7a659b46493e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:55.113775 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e5d494-5db7-44a3-a886-7a659b46493e None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:55.114051 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e5d494-5db7-44a3-a886-7a659b46493e None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:55.114319 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e5d494-5db7-44a3-a886-7a659b46493e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:55.115122 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e5d494-5db7-44a3-a886-7a659b46493e None admin] RBAC: Adding query filter params (interface=public, service_id=adffb71bc5cc43f7bbd9603d2c7f77b7, region_id=RegionOne) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:55.115456 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e5d494-5db7-44a3-a886-7a659b46493e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:55.116809 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e5d494-5db7-44a3-a886-7a659b46493e None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:55.123214 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 385/776] 10.222.0.61 () {56 vars in 1251 bytes} [Fri Aug 26 12:01:54 2022] GET /identity/v3/endpoints?service_id=adffb71bc5cc43f7bbd9603d2c7f77b7&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:55.124078 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e5d494-5db7-44a3-a886-7a659b46493e None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:55.133923 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9926422b-9fd8-4c53-9ff9-58f0db5cd74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:55.134560 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9926422b-9fd8-4c53-9ff9-58f0db5cd74b None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:55.134885 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9926422b-9fd8-4c53-9ff9-58f0db5cd74b None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:55.135175 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9926422b-9fd8-4c53-9ff9-58f0db5cd74b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:55.146093 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9926422b-9fd8-4c53-9ff9-58f0db5cd74b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:01:55.151465 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-9926422b-9fd8-4c53-9ff9-58f0db5cd74b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 12:01:55.151465 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:55.151465 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:55.151465 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:55.151465 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:55.151465 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:55.151465 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:55.151465 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:55.151465 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:55.151465 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:55.151465 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:55.151465 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:55.151465 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:01:55.151465 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:01:55.151465 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:01:55.151465 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:01:55.152427 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:01:55.152427 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:01:55.152427 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:01:55.152427 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:01:55.152427 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:55.152427 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:55.152427 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:55.152427 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:55.152427 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:55.152427 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:55.152427 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:55.152427 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:55.152427 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:55.152427 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:55.152427 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:55.152427 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:55.153366 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:55.153366 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:55.153366 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:55.153366 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:55.153366 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:01:55.153366 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:01:55.153366 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:01:55.153366 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:01:55.153366 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:01:55.153366 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:01:55.153366 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 12:01:55.153366 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:55.153366 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 392/777] 10.222.0.61 () {56 vars in 1096 bytes} [Fri Aug 26 12:01:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:55.153366 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9926422b-9fd8-4c53-9ff9-58f0db5cd74b None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:55.237343 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e5d494-5db7-44a3-a886-7a659b46493e None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:55.262475 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9926422b-9fd8-4c53-9ff9-58f0db5cd74b None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:55.269425 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:55.274978 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b502d41-7c49-4019-aab9-dfca364b77c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:55.275785 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b502d41-7c49-4019-aab9-dfca364b77c1 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:55.276209 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b502d41-7c49-4019-aab9-dfca364b77c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:55.276641 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b502d41-7c49-4019-aab9-dfca364b77c1 None admin] PATH_INFO: `/v3/services` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:55.277397 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b502d41-7c49-4019-aab9-dfca364b77c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:55.279636 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b502d41-7c49-4019-aab9-dfca364b77c1 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:55.285100 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 386/778] 10.222.0.61 () {56 vars in 1087 bytes} [Fri Aug 26 12:01:55 2022] GET /identity/v3/services => generated 772 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:55.286073 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b502d41-7c49-4019-aab9-dfca364b77c1 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:55.311832 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969ae718-72c5-4813-8c75-9c0998f23426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:55.312761 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:55.313959 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969ae718-72c5-4813-8c75-9c0998f23426 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:55.314443 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969ae718-72c5-4813-8c75-9c0998f23426 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:55.314873 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969ae718-72c5-4813-8c75-9c0998f23426 None admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:55.315727 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969ae718-72c5-4813-8c75-9c0998f23426 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:55.316179 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969ae718-72c5-4813-8c75-9c0998f23426 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:55.317755 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969ae718-72c5-4813-8c75-9c0998f23426 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:55.334877 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 393/779] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:01:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:55.337123 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969ae718-72c5-4813-8c75-9c0998f23426 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:55.428031 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b502d41-7c49-4019-aab9-dfca364b77c1 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:55.474522 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969ae718-72c5-4813-8c75-9c0998f23426 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:55.482470 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447763f4-2f87-495f-a5d5-098cbda3905a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:55.483094 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447763f4-2f87-495f-a5d5-098cbda3905a None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:55.483479 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447763f4-2f87-495f-a5d5-098cbda3905a None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:55.483801 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447763f4-2f87-495f-a5d5-098cbda3905a None admin] PATH_INFO: `/v3/services` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:55.485508 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447763f4-2f87-495f-a5d5-098cbda3905a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:55.487886 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447763f4-2f87-495f-a5d5-098cbda3905a None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:55.495365 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 387/780] 10.222.0.61 () {56 vars in 1087 bytes} [Fri Aug 26 12:01:55 2022] GET /identity/v3/services => generated 772 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:55.496211 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447763f4-2f87-495f-a5d5-098cbda3905a None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:55.524303 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f408ca50-3802-4ed6-bf41-f52dd26407ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:55.524938 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f408ca50-3802-4ed6-bf41-f52dd26407ff None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:55.525224 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f408ca50-3802-4ed6-bf41-f52dd26407ff None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:55.525499 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f408ca50-3802-4ed6-bf41-f52dd26407ff None admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:55.526328 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f408ca50-3802-4ed6-bf41-f52dd26407ff None admin] RBAC: Adding query filter params (name=admin) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:55.526640 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f408ca50-3802-4ed6-bf41-f52dd26407ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:55.528699 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f408ca50-3802-4ed6-bf41-f52dd26407ff None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:55.547481 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 394/781] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:01:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:55.549034 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe07e78-d59c-4773-9210-763072942613 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:55.549324 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe07e78-d59c-4773-9210-763072942613 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:55.549619 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe07e78-d59c-4773-9210-763072942613 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:55.552238 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 395/782] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:01:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:01:55.553055 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe07e78-d59c-4773-9210-763072942613 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:55.592943 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:55.618130 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:55.654013 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447763f4-2f87-495f-a5d5-098cbda3905a None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:55.695407 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe07e78-d59c-4773-9210-763072942613 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:55.700207 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cd48a1-8426-44e1-a6c9-9cf0b43b5e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:55.700889 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd48a1-8426-44e1-a6c9-9cf0b43b5e9e None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:55.701172 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd48a1-8426-44e1-a6c9-9cf0b43b5e9e None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:55.701451 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd48a1-8426-44e1-a6c9-9cf0b43b5e9e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:55.710226 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cd48a1-8426-44e1-a6c9-9cf0b43b5e9e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:55.711755 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cd48a1-8426-44e1-a6c9-9cf0b43b5e9e None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:55.725832 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-71cd48a1-8426-44e1-a6c9-9cf0b43b5e9e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:01:55.725832 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:55.725832 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:55.725832 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:55.725832 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:55.725832 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:55.725832 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:55.725832 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:55.725832 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:55.725832 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:55.725832 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:55.725832 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:55.725832 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 12:01:55.725832 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 12:01:55.725832 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 12:01:55.726781 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 12:01:55.726781 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:55.726781 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:55.726781 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:55.726781 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:55.726781 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:55.726781 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:55.726781 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:55.726781 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:55.726781 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:55.726781 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:55.726781 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:55.726781 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:55.726781 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:55.726781 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:55.726781 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:55.726781 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:55.727754 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 12:01:55.727754 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 12:01:55.727754 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 12:01:55.727754 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 12:01:55.727754 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 12:01:55.727754 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 12:01:55.727754 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:01:55.727754 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:55.727754 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 388/783] 10.222.0.61 () {56 vars in 1111 bytes} [Fri Aug 26 12:01:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:55.728531 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3605c10c-1293-4ac7-9491-700b6f3c5de1 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:55.728759 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3605c10c-1293-4ac7-9491-700b6f3c5de1 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:55.729010 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3605c10c-1293-4ac7-9491-700b6f3c5de1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:55.738311 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc96912-483e-4fef-b9b8-ef1b95ab79ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:55.739307 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc96912-483e-4fef-b9b8-ef1b95ab79ee None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:55.739788 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc96912-483e-4fef-b9b8-ef1b95ab79ee None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:55.740234 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc96912-483e-4fef-b9b8-ef1b95ab79ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:55.754231 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc96912-483e-4fef-b9b8-ef1b95ab79ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:55.760337 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc96912-483e-4fef-b9b8-ef1b95ab79ee None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:55.762318 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-3605c10c-1293-4ac7-9491-700b6f3c5de1 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:01:55.764648 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-3605c10c-1293-4ac7-9491-700b6f3c5de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:01:55.777612 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-4cc96912-483e-4fef-b9b8-ef1b95ab79ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:55.777612 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:55.777612 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:55.777612 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:55.777612 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:55.777612 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:55.777612 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:55.777612 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:55.777612 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:55.777612 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:55.777612 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:55.777612 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:55.777612 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:01:55.777612 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:01:55.777612 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:01:55.777612 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:01:55.779620 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:55.779620 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:55.779620 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:55.779620 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:55.779620 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:55.779620 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:55.779620 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:55.779620 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:55.779620 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:55.779620 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:55.779620 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:55.779620 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:55.779620 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:55.779620 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:55.779620 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:55.779620 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:55.779620 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:01:55.780518 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:01:55.780518 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:55.780518 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:55.780518 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 396/784] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:01:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 12:01:55.780765 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc96912-483e-4fef-b9b8-ef1b95ab79ee None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:55.847871 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 389/785] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:01:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 12:01:55.848657 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3605c10c-1293-4ac7-9491-700b6f3c5de1 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:55.899707 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:55.928921 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc96912-483e-4fef-b9b8-ef1b95ab79ee None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:55.931872 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:55.970924 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3605c10c-1293-4ac7-9491-700b6f3c5de1 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:56.008347 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a62208-33fc-4f4c-89fb-c428614dacd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:56.008969 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a62208-33fc-4f4c-89fb-c428614dacd4 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:56.009250 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a62208-33fc-4f4c-89fb-c428614dacd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:56.009527 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a62208-33fc-4f4c-89fb-c428614dacd4 None admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:56.010172 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a62208-33fc-4f4c-89fb-c428614dacd4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:56.010500 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a62208-33fc-4f4c-89fb-c428614dacd4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:56.012828 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a62208-33fc-4f4c-89fb-c428614dacd4 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:56.025591 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 390/786] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:01:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:56.028004 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e6ba41-bf04-48a0-b29f-106184108cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:56.030183 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6ba41-bf04-48a0-b29f-106184108cf3 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:56.030590 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6ba41-bf04-48a0-b29f-106184108cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:56.030983 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e6ba41-bf04-48a0-b29f-106184108cf3 None admin] PATH_INFO: `/v3/projects` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:56.031474 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a62208-33fc-4f4c-89fb-c428614dacd4 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:56.032007 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e6ba41-bf04-48a0-b29f-106184108cf3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:56.032435 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e6ba41-bf04-48a0-b29f-106184108cf3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:56.034357 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e6ba41-bf04-48a0-b29f-106184108cf3 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:56.053104 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 397/787] 10.222.0.61 () {56 vars in 1148 bytes} [Fri Aug 26 12:01:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:56.061607 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e6ba41-bf04-48a0-b29f-106184108cf3 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:56.210590 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a62208-33fc-4f4c-89fb-c428614dacd4 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:56.240909 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:56.252750 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89df070-96fc-4314-856f-f3cfd0e01ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:56.253612 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89df070-96fc-4314-856f-f3cfd0e01ebc None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:56.254007 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89df070-96fc-4314-856f-f3cfd0e01ebc None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:56.254349 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89df070-96fc-4314-856f-f3cfd0e01ebc None admin] PATH_INFO: `/v3/users/swift` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:56.264003 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89df070-96fc-4314-856f-f3cfd0e01ebc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:56.265850 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89df070-96fc-4314-856f-f3cfd0e01ebc None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:56.271471 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e6ba41-bf04-48a0-b29f-106184108cf3 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:56.278173 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-d89df070-96fc-4314-856f-f3cfd0e01ebc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 26 12:01:56.278173 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:56.278173 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:56.278173 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:56.278173 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:56.278173 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:56.278173 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:56.278173 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:56.278173 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:56.278173 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:56.278173 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:56.278173 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:56.278173 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 12:01:56.278173 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 12:01:56.278173 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 12:01:56.278173 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 12:01:56.279934 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:56.279934 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:56.279934 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 12:01:56.279934 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:01:56.279934 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 12:01:56.279934 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:01:56.279934 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:56.279934 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:56.279934 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:56.279934 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:56.279934 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:56.279934 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:56.279934 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:56.279934 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:56.279934 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:56.279934 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:56.279934 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:56.279934 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:56.281816 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:56.281816 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:56.281816 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 12:01:56.281816 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 12:01:56.281816 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 12:01:56.281816 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 12:01:56.281816 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 12:01:56.281816 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 12:01:56.281816 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 26 12:01:56.281816 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:56.281816 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 391/788] 10.222.0.61 () {56 vars in 1096 bytes} [Fri Aug 26 12:01:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:56.281816 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89df070-96fc-4314-856f-f3cfd0e01ebc None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:56.307668 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c7b3d3-8f5b-49bf-977e-de0ec094b84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:56.308364 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c7b3d3-8f5b-49bf-977e-de0ec094b84c None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:56.308642 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c7b3d3-8f5b-49bf-977e-de0ec094b84c None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:56.309016 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c7b3d3-8f5b-49bf-977e-de0ec094b84c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:56.310349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c7b3d3-8f5b-49bf-977e-de0ec094b84c None admin] RBAC: Adding query filter params (role.id=519347d2bd2c4dc1aadb0b9f8c3c8e3d, scope.project.id=ecd68423a83a427681d44dbf266fce6e, user.id=755b04cae45746569bfbf92f2f213a58) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:56.310650 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c7b3d3-8f5b-49bf-977e-de0ec094b84c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:56.312202 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c7b3d3-8f5b-49bf-977e-de0ec094b84c None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:56.318049 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 398/789] 10.222.0.61 () {56 vars in 1374 bytes} [Fri Aug 26 12:01:56 2022] GET /identity/v3/role_assignments?user.id=755b04cae45746569bfbf92f2f213a58&scope.project.id=ecd68423a83a427681d44dbf266fce6e&role.id=519347d2bd2c4dc1aadb0b9f8c3c8e3d => generated 259 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:56.318922 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c7b3d3-8f5b-49bf-977e-de0ec094b84c None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:56.323731 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:56.456991 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89df070-96fc-4314-856f-f3cfd0e01ebc None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:56.499915 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c7b3d3-8f5b-49bf-977e-de0ec094b84c None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:56.536512 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a347bc9-e0b5-4136-8f42-36eb883b43a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:56.537709 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a347bc9-e0b5-4136-8f42-36eb883b43a7 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:56.538145 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a347bc9-e0b5-4136-8f42-36eb883b43a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:56.538588 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a347bc9-e0b5-4136-8f42-36eb883b43a7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:56.553489 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fca383-e62d-4968-b7fb-a113011e6cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:56.554160 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fca383-e62d-4968-b7fb-a113011e6cf8 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:56.554375 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fca383-e62d-4968-b7fb-a113011e6cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:56.554647 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fca383-e62d-4968-b7fb-a113011e6cf8 None admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:56.555423 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fca383-e62d-4968-b7fb-a113011e6cf8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:56.555730 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fca383-e62d-4968-b7fb-a113011e6cf8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:56.556297 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a347bc9-e0b5-4136-8f42-36eb883b43a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:01:56.557174 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fca383-e62d-4968-b7fb-a113011e6cf8 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:56.564202 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-1a347bc9-e0b5-4136-8f42-36eb883b43a7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 12:01:56.564202 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:56.564202 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:56.564202 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:56.564202 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:56.564202 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:56.564202 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:56.564202 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:56.564202 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:56.564202 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:56.564202 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:56.564202 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:56.564202 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:01:56.564202 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:01:56.564202 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:01:56.564202 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:01:56.565396 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:01:56.565396 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:01:56.565396 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:01:56.565396 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:01:56.565396 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:56.565396 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:56.565396 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:56.565396 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:56.565396 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:56.565396 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:56.565396 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:56.565396 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:56.565396 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:56.565396 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:56.565396 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:56.565396 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:56.566462 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:56.566462 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:56.566462 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:56.566462 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:56.566462 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:01:56.566462 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:01:56.566462 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:01:56.566462 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:01:56.566462 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:01:56.566462 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:01:56.566462 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 12:01:56.566462 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:56.568709 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 392/790] 10.222.0.61 () {56 vars in 1096 bytes} [Fri Aug 26 12:01:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:56.574600 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a347bc9-e0b5-4136-8f42-36eb883b43a7 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:56.605966 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 399/791] 10.222.0.61 () {56 vars in 1135 bytes} [Fri Aug 26 12:01:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:56.612254 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fca383-e62d-4968-b7fb-a113011e6cf8 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:56.639647 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:56.695368 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a347bc9-e0b5-4136-8f42-36eb883b43a7 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:56.734758 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f09a90c-89b3-4a46-b529-9c50c2fff807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:56.735949 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f09a90c-89b3-4a46-b529-9c50c2fff807 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:56.736486 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f09a90c-89b3-4a46-b529-9c50c2fff807 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:56.736961 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f09a90c-89b3-4a46-b529-9c50c2fff807 None admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:56.738292 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f09a90c-89b3-4a46-b529-9c50c2fff807 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:56.738866 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f09a90c-89b3-4a46-b529-9c50c2fff807 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:56.748355 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f09a90c-89b3-4a46-b529-9c50c2fff807 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:56.759244 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 393/792] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:01:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:01:56.769687 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f09a90c-89b3-4a46-b529-9c50c2fff807 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:56.812479 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:56.902467 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fca383-e62d-4968-b7fb-a113011e6cf8 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:56.958843 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f09a90c-89b3-4a46-b529-9c50c2fff807 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:56.970236 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:56.973967 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62bd471-1f45-49a8-b0b6-baef76180c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:56.974614 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62bd471-1f45-49a8-b0b6-baef76180c9b None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:56.974897 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62bd471-1f45-49a8-b0b6-baef76180c9b None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:56.975483 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62bd471-1f45-49a8-b0b6-baef76180c9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:56.997902 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fa3d98-2ae5-4dda-a484-81fcbb57aaed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:56.998751 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa3d98-2ae5-4dda-a484-81fcbb57aaed None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:56.999193 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa3d98-2ae5-4dda-a484-81fcbb57aaed None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:56.999622 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa3d98-2ae5-4dda-a484-81fcbb57aaed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:57.000260 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62bd471-1f45-49a8-b0b6-baef76180c9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:57.001771 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62bd471-1f45-49a8-b0b6-baef76180c9b None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:57.011415 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fa3d98-2ae5-4dda-a484-81fcbb57aaed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:57.013811 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fa3d98-2ae5-4dda-a484-81fcbb57aaed None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:57.028367 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-25fa3d98-2ae5-4dda-a484-81fcbb57aaed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:57.028367 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:57.028367 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:57.028367 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:57.028367 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:57.028367 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:57.028367 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:57.028367 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:57.028367 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:57.028367 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:57.028367 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:57.028367 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:57.028367 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:01:57.028367 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:01:57.028367 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:01:57.028367 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:01:57.029515 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:57.029515 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:57.029515 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:57.029515 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:57.029515 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:57.029515 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:57.029515 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:57.029515 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:57.029515 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:57.029515 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:57.029515 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:57.029515 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:57.029515 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:57.029515 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:57.029515 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:57.029515 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:57.029515 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:01:57.030617 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:01:57.030617 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:57.030617 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:57.032286 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 394/793] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:01:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:57.034138 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-e62bd471-1f45-49a8-b0b6-baef76180c9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:57.034138 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:57.034138 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:57.034138 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:57.034138 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:57.034138 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:57.034138 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:57.034138 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:57.034138 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:57.034138 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:57.034138 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:57.034138 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:57.034138 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:01:57.034138 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:01:57.034138 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:01:57.034138 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:01:57.035194 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:57.035194 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:57.035194 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:57.035194 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:57.035194 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:57.035194 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:57.035194 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:57.035194 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:57.035194 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:57.035194 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:57.035194 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:57.035194 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:57.035194 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:57.035194 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:57.035194 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:57.035194 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:57.035194 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:01:57.036308 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:01:57.036308 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:57.036308 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:57.046338 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 400/794] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:01:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 424 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:57.046493 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fa3d98-2ae5-4dda-a484-81fcbb57aaed None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:57.048888 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62bd471-1f45-49a8-b0b6-baef76180c9b None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:57.162055 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fa3d98-2ae5-4dda-a484-81fcbb57aaed None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:57.223984 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8127816e-6189-414c-8b30-5a7104c65f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:57.224619 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8127816e-6189-414c-8b30-5a7104c65f24 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:57.224900 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8127816e-6189-414c-8b30-5a7104c65f24 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:57.225167 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8127816e-6189-414c-8b30-5a7104c65f24 None admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:57.225802 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8127816e-6189-414c-8b30-5a7104c65f24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:57.226099 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8127816e-6189-414c-8b30-5a7104c65f24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:57.227510 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8127816e-6189-414c-8b30-5a7104c65f24 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:57.249755 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62bd471-1f45-49a8-b0b6-baef76180c9b None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:57.257762 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 395/795] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:01:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:57.270630 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:57.277263 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8127816e-6189-414c-8b30-5a7104c65f24 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:57.295609 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f335ab-0163-473e-a8c3-64965073d84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:57.296536 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f335ab-0163-473e-a8c3-64965073d84e None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:57.297020 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f335ab-0163-473e-a8c3-64965073d84e None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:57.297519 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f335ab-0163-473e-a8c3-64965073d84e None admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:57.298362 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f335ab-0163-473e-a8c3-64965073d84e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:57.298868 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f335ab-0163-473e-a8c3-64965073d84e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:57.300497 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f335ab-0163-473e-a8c3-64965073d84e None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:57.314923 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 401/796] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:01:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:01:57.320688 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f335ab-0163-473e-a8c3-64965073d84e None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:57.339594 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:57.446509 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8127816e-6189-414c-8b30-5a7104c65f24 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:57.499749 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f335ab-0163-473e-a8c3-64965073d84e None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:57.538805 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e414082-371e-49ff-8986-f836e9d7eef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:57.539454 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e414082-371e-49ff-8986-f836e9d7eef7 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:57.539727 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e414082-371e-49ff-8986-f836e9d7eef7 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:57.539927 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c2731b-b2ac-4287-92c2-b1f3ca0cfba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:57.540142 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e414082-371e-49ff-8986-f836e9d7eef7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:57.540552 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c2731b-b2ac-4287-92c2-b1f3ca0cfba1 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:57.540830 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c2731b-b2ac-4287-92c2-b1f3ca0cfba1 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:57.541099 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c2731b-b2ac-4287-92c2-b1f3ca0cfba1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:57.548882 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e414082-371e-49ff-8986-f836e9d7eef7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:57.550634 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e414082-371e-49ff-8986-f836e9d7eef7 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:57.554076 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c2731b-b2ac-4287-92c2-b1f3ca0cfba1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:57.563566 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-1e414082-371e-49ff-8986-f836e9d7eef7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 12:01:57.563566 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:57.563566 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:57.563566 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:57.563566 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:57.563566 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:57.563566 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:57.563566 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:57.563566 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:57.563566 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:57.563566 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:57.563566 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:57.563566 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 12:01:57.563566 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 12:01:57.563566 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 12:01:57.563566 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 12:01:57.564534 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:57.564534 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:57.564534 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 12:01:57.564534 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:01:57.564534 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 12:01:57.564534 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:01:57.564534 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:57.564534 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:57.564534 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:57.564534 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:57.564534 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:57.564534 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:57.564534 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:57.564534 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:57.564534 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:57.564534 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:57.564534 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:57.564534 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:57.565511 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:57.565511 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:57.565511 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 12:01:57.565511 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 12:01:57.565511 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 12:01:57.565511 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 12:01:57.565511 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 12:01:57.565511 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 12:01:57.565511 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 12:01:57.565511 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:57.566065 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c2731b-b2ac-4287-92c2-b1f3ca0cfba1 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:57.566138 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 402/797] 10.222.0.61 () {56 vars in 1093 bytes} [Fri Aug 26 12:01:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 12:01:57.572130 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e414082-371e-49ff-8986-f836e9d7eef7 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:57.581454 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-39c2731b-b2ac-4287-92c2-b1f3ca0cfba1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:01:57.581454 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:57.581454 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:57.581454 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:57.581454 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:57.581454 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:57.581454 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:57.581454 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:57.581454 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:57.581454 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:57.581454 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:57.581454 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:57.581454 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 12:01:57.581454 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 12:01:57.581454 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 12:01:57.582474 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 12:01:57.582474 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:57.582474 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:57.582474 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:57.582474 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:57.582474 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:57.582474 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:57.582474 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:57.582474 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:57.582474 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:57.582474 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:57.582474 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:57.582474 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:57.582474 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:57.582474 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:57.582474 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:57.582474 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:57.583801 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 12:01:57.583801 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 12:01:57.583801 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 12:01:57.583801 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 12:01:57.583801 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 12:01:57.583801 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 12:01:57.583801 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:01:57.583801 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:57.585887 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 396/798] 10.222.0.61 () {56 vars in 1111 bytes} [Fri Aug 26 12:01:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:57.587401 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32217c7f-8035-458e-a18e-087d1cc4b479 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:57.587891 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32217c7f-8035-458e-a18e-087d1cc4b479 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:57.588364 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32217c7f-8035-458e-a18e-087d1cc4b479 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:57.593979 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 397/799] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:01:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:01:57.594986 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32217c7f-8035-458e-a18e-087d1cc4b479 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:57.613981 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:57.695874 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e414082-371e-49ff-8986-f836e9d7eef7 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:57.706948 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:57.735752 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32217c7f-8035-458e-a18e-087d1cc4b479 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:57.739754 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a13ac6-fe85-4fba-b6a3-19d823c12fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:57.740399 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a13ac6-fe85-4fba-b6a3-19d823c12fe3 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:57.740678 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a13ac6-fe85-4fba-b6a3-19d823c12fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:57.740950 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a13ac6-fe85-4fba-b6a3-19d823c12fe3 None admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:57.741691 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a13ac6-fe85-4fba-b6a3-19d823c12fe3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:57.741995 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a13ac6-fe85-4fba-b6a3-19d823c12fe3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:57.743459 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a13ac6-fe85-4fba-b6a3-19d823c12fe3 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:57.769234 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 403/800] 10.222.0.61 () {56 vars in 1133 bytes} [Fri Aug 26 12:01:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:57.770774 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8e8907-b874-4ddf-a13d-4bfc07ffa285 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:57.771063 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8e8907-b874-4ddf-a13d-4bfc07ffa285 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:57.771370 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8e8907-b874-4ddf-a13d-4bfc07ffa285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:57.786565 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aee89e3-e73d-4f3e-9289-c9babff121b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:57.786951 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee89e3-e73d-4f3e-9289-c9babff121b0 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:57.787322 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee89e3-e73d-4f3e-9289-c9babff121b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:57.787601 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee89e3-e73d-4f3e-9289-c9babff121b0 None admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:57.788345 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aee89e3-e73d-4f3e-9289-c9babff121b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:57.788652 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aee89e3-e73d-4f3e-9289-c9babff121b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:57.790138 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aee89e3-e73d-4f3e-9289-c9babff121b0 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:57.794608 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-9f8e8907-b874-4ddf-a13d-4bfc07ffa285 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:01:57.796295 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-9f8e8907-b874-4ddf-a13d-4bfc07ffa285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:01:57.803242 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 398/801] 10.222.0.61 () {56 vars in 1148 bytes} [Fri Aug 26 12:01:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:57.804254 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aee89e3-e73d-4f3e-9289-c9babff121b0 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:57.871125 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 404/802] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:01:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 12:01:57.872549 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8e8907-b874-4ddf-a13d-4bfc07ffa285 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:57.899083 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:57.925981 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aee89e3-e73d-4f3e-9289-c9babff121b0 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:57.964813 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f642af-82d1-44d0-82ba-7e07b3bc9e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:57.966752 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f642af-82d1-44d0-82ba-7e07b3bc9e68 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:57.967930 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f642af-82d1-44d0-82ba-7e07b3bc9e68 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:57.968225 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f642af-82d1-44d0-82ba-7e07b3bc9e68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:57.982659 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:57.987608 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8e8907-b874-4ddf-a13d-4bfc07ffa285 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:58.036221 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53706eff-5af4-43fb-b013-de7fd7c3ba25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:58.037949 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53706eff-5af4-43fb-b013-de7fd7c3ba25 None admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:58.038385 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53706eff-5af4-43fb-b013-de7fd7c3ba25 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:58.038805 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53706eff-5af4-43fb-b013-de7fd7c3ba25 None admin] PATH_INFO: `/v3/projects/ecd68423a83a427681d44dbf266fce6e/users/ead95a244d11432d9fe9f718e1a75f38/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:58.042044 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53706eff-5af4-43fb-b013-de7fd7c3ba25 None admin] RBAC: Authorizing `identity:create_grant(project_id=ecd68423a83a427681d44dbf266fce6e, user_id=ead95a244d11432d9fe9f718e1a75f38, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:58.044747 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53706eff-5af4-43fb-b013-de7fd7c3ba25 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:58.047770 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f642af-82d1-44d0-82ba-7e07b3bc9e68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:58.049578 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f642af-82d1-44d0-82ba-7e07b3bc9e68 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:58.064354 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 405/803] 10.222.0.61 () {58 vars in 1439 bytes} [Fri Aug 26 12:01:57 2022] PUT /identity/v3/projects/ecd68423a83a427681d44dbf266fce6e/users/ead95a244d11432d9fe9f718e1a75f38/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:01:58.066056 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc50d0-edd9-4478-9e48-7bb72de6cb53 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:58.066479 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc50d0-edd9-4478-9e48-7bb72de6cb53 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:58.066897 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc50d0-edd9-4478-9e48-7bb72de6cb53 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:58.068636 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 406/804] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:01:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:01:58.082972 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9690c885-fa2f-48c9-8e32-91f0e9e06262 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:58.083513 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9690c885-fa2f-48c9-8e32-91f0e9e06262 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:58.083998 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9690c885-fa2f-48c9-8e32-91f0e9e06262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:58.127740 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-9690c885-fa2f-48c9-8e32-91f0e9e06262 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:01:58.129171 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-9690c885-fa2f-48c9-8e32-91f0e9e06262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:01:58.222806 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 399/805] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:01:57 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 12:01:58.223976 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f642af-82d1-44d0-82ba-7e07b3bc9e68 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:58.280569 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:58.343228 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 407/806] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:01:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 12:01:58.344092 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9690c885-fa2f-48c9-8e32-91f0e9e06262 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:58.345836 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f642af-82d1-44d0-82ba-7e07b3bc9e68 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:58.360128 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:58.398596 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938fba3e-09bc-47d9-8a20-44fa7ab21fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:58.399265 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938fba3e-09bc-47d9-8a20-44fa7ab21fb7 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:58.399551 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938fba3e-09bc-47d9-8a20-44fa7ab21fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:58.399823 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938fba3e-09bc-47d9-8a20-44fa7ab21fb7 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:58.400412 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938fba3e-09bc-47d9-8a20-44fa7ab21fb7 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:58.401788 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938fba3e-09bc-47d9-8a20-44fa7ab21fb7 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:58.413556 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-938fba3e-09bc-47d9-8a20-44fa7ab21fb7 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 12:01:58.413556 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:58.413556 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 12:01:58.413556 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 12:01:58.413556 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 12:01:58.413556 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 12:01:58.413556 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 12:01:58.413556 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 12:01:58.413556 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 12:01:58.413556 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:58.413556 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 12:01:58.413556 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:58.413556 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:58.413556 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:58.413556 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:58.413556 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:58.413556 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:58.414803 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:58.414803 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:58.414803 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:58.414803 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:58.414803 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:58.414803 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:58.414803 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 12:01:58.414803 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 12:01:58.414803 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 12:01:58.414803 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 12:01:58.414803 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:58.414803 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:58.414803 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:58.414803 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:58.414803 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:58.414803 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:58.415744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:58.415744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:58.415744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:58.415744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:58.415744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:58.415744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:58.415744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:58.415744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:58.415744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:58.415744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:58.415744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 12:01:58.415744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 12:01:58.415744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 26 12:01:58.415744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:58.419373 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 400/807] 10.222.0.61 () {56 vars in 1129 bytes} [Fri Aug 26 12:01:58 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:58.421431 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938fba3e-09bc-47d9-8a20-44fa7ab21fb7 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:58.474255 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9690c885-fa2f-48c9-8e32-91f0e9e06262 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:58.526930 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d2d479-3941-46f5-ac0a-ae674a519f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:58.527843 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d2d479-3941-46f5-ac0a-ae674a519f2a None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:58.528350 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d2d479-3941-46f5-ac0a-ae674a519f2a None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:58.529120 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d2d479-3941-46f5-ac0a-ae674a519f2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:58.531692 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938fba3e-09bc-47d9-8a20-44fa7ab21fb7 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:58.539476 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:58.542990 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d2d479-3941-46f5-ac0a-ae674a519f2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:58.545537 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d2d479-3941-46f5-ac0a-ae674a519f2a None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:58.565264 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-60d2d479-3941-46f5-ac0a-ae674a519f2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:58.565264 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:58.565264 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:58.565264 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:58.565264 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:58.565264 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:58.565264 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:58.565264 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:58.565264 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:58.565264 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:58.565264 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:58.565264 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:58.565264 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:01:58.565264 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:01:58.565264 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:01:58.565264 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:01:58.567458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:58.567458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:58.567458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:58.567458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:58.567458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:58.567458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:58.567458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:58.567458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:58.567458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:58.567458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:58.567458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:58.567458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:58.567458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:58.567458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:58.567458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:58.567458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:58.567458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:01:58.569491 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:01:58.569491 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:01:58.569491 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:58.569491 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 408/808] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:01:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:58.575235 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d2d479-3941-46f5-ac0a-ae674a519f2a None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:58.605821 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a51790d-4472-47dd-bbce-cde57c54292e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:58.606893 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a51790d-4472-47dd-bbce-cde57c54292e None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:58.607172 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a51790d-4472-47dd-bbce-cde57c54292e None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:58.607472 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a51790d-4472-47dd-bbce-cde57c54292e None admin] PATH_INFO: `/v3/services` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:58.608108 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a51790d-4472-47dd-bbce-cde57c54292e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:58.608408 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a51790d-4472-47dd-bbce-cde57c54292e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:58.609919 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a51790d-4472-47dd-bbce-cde57c54292e None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:58.624421 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 401/809] 10.222.0.61 () {56 vars in 1124 bytes} [Fri Aug 26 12:01:58 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:58.629972 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a51790d-4472-47dd-bbce-cde57c54292e None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:58.645435 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:58.703103 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d2d479-3941-46f5-ac0a-ae674a519f2a None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:58.749246 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a51790d-4472-47dd-bbce-cde57c54292e None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:58.754614 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930ccfca-542a-4663-866d-3cd4be6e5e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:58.755246 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ccfca-542a-4663-866d-3cd4be6e5e0c None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:58.755534 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ccfca-542a-4663-866d-3cd4be6e5e0c None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:58.755807 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ccfca-542a-4663-866d-3cd4be6e5e0c None admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:58.756567 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930ccfca-542a-4663-866d-3cd4be6e5e0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:58.756846 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930ccfca-542a-4663-866d-3cd4be6e5e0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:58.758858 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930ccfca-542a-4663-866d-3cd4be6e5e0c None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:58.771885 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 409/810] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:01:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:58.773137 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c4f81-1cc8-494f-ba8e-e6af07dd1890 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:58.773424 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c4f81-1cc8-494f-ba8e-e6af07dd1890 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:58.773701 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c4f81-1cc8-494f-ba8e-e6af07dd1890 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:58.775060 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 410/811] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:01:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:01:58.775610 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96c4f81-1cc8-494f-ba8e-e6af07dd1890 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:58.790205 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc6e7d8-6354-4c6a-933d-f090fc57bc04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:58.790812 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc6e7d8-6354-4c6a-933d-f090fc57bc04 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:58.791131 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc6e7d8-6354-4c6a-933d-f090fc57bc04 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:58.791382 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc6e7d8-6354-4c6a-933d-f090fc57bc04 None admin] PATH_INFO: `/v3/services` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:58.792169 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc6e7d8-6354-4c6a-933d-f090fc57bc04 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:58.792365 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc6e7d8-6354-4c6a-933d-f090fc57bc04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:58.794351 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc6e7d8-6354-4c6a-933d-f090fc57bc04 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:58.800356 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 402/812] 10.222.0.61 () {56 vars in 1124 bytes} [Fri Aug 26 12:01:58 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:01:58.804389 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc6e7d8-6354-4c6a-933d-f090fc57bc04 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:58.840568 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:58.889642 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96c4f81-1cc8-494f-ba8e-e6af07dd1890 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:58.912481 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc6e7d8-6354-4c6a-933d-f090fc57bc04 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:58.920899 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:58.936912 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfc6492-3b22-4e08-8317-46f7d711f627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:58.937819 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc6492-3b22-4e08-8317-46f7d711f627 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:58.938104 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc6492-3b22-4e08-8317-46f7d711f627 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:58.938535 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc6492-3b22-4e08-8317-46f7d711f627 None admin] PATH_INFO: `/v3/services/network` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:58.939474 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfc6492-3b22-4e08-8317-46f7d711f627 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:58.941270 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfc6492-3b22-4e08-8317-46f7d711f627 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:58.946565 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5a3cdc-6491-4b07-bc74-4f42f1427389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:58.947181 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a3cdc-6491-4b07-bc74-4f42f1427389 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:58.947488 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a3cdc-6491-4b07-bc74-4f42f1427389 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:58.947759 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a3cdc-6491-4b07-bc74-4f42f1427389 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:58.951323 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-7bfc6492-3b22-4e08-8317-46f7d711f627 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 12:01:58.951323 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:58.951323 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 12:01:58.951323 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 12:01:58.951323 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 12:01:58.951323 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 12:01:58.951323 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 12:01:58.951323 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 12:01:58.951323 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 12:01:58.951323 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:58.951323 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 12:01:58.951323 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:58.951323 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:58.951323 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:58.951323 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:58.951323 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:58.951323 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:58.952317 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:58.952317 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:58.952317 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:58.952317 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:58.952317 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:58.952317 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:58.952317 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 12:01:58.952317 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 12:01:58.952317 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 12:01:58.952317 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 12:01:58.952317 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:58.952317 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:58.952317 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:58.952317 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:58.952317 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:58.952317 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:58.953838 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:58.953838 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:58.953838 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:58.953838 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:58.953838 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:58.953838 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:58.953838 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:58.953838 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:58.953838 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:58.953838 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:58.953838 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 12:01:58.953838 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 12:01:58.953838 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 26 12:01:58.953838 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:58.953838 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 411/813] 10.222.0.61 () {56 vars in 1111 bytes} [Fri Aug 26 12:01:58 2022] GET /identity/v3/services/network => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:58.954655 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489ee5ce-1d40-464b-a71e-5670b5d32779 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:58.954655 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489ee5ce-1d40-464b-a71e-5670b5d32779 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:58.954788 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5a3cdc-6491-4b07-bc74-4f42f1427389 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:58.954868 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489ee5ce-1d40-464b-a71e-5670b5d32779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:58.955961 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5a3cdc-6491-4b07-bc74-4f42f1427389 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:58.974996 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-489ee5ce-1d40-464b-a71e-5670b5d32779 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:01:58.976276 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-489ee5ce-1d40-464b-a71e-5670b5d32779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:01:59.060040 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 412/814] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:01:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 12:01:59.061410 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489ee5ce-1d40-464b-a71e-5670b5d32779 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:59.083944 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-6f5a3cdc-6491-4b07-bc74-4f42f1427389 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:01:59.083944 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:59.083944 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:59.083944 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:59.083944 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:59.083944 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:59.083944 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:59.083944 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:59.083944 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:59.083944 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:59.083944 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:59.083944 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:59.083944 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 12:01:59.083944 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 12:01:59.083944 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 12:01:59.085269 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 12:01:59.085269 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:59.085269 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:59.085269 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:59.085269 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:59.085269 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:59.085269 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:59.085269 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:59.085269 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:59.085269 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:59.085269 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:59.085269 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:59.085269 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:59.085269 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:59.085269 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:59.085269 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:59.085269 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:59.086637 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 12:01:59.086637 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 12:01:59.086637 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 12:01:59.086637 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 12:01:59.086637 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 12:01:59.086637 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 12:01:59.086637 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:01:59.086637 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:01:59.089577 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 403/815] 10.222.0.61 () {56 vars in 1111 bytes} [Fri Aug 26 12:01:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:59.090405 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5a3cdc-6491-4b07-bc74-4f42f1427389 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:59.178938 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489ee5ce-1d40-464b-a71e-5670b5d32779 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:59.190060 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:59.197529 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5a3cdc-6491-4b07-bc74-4f42f1427389 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:59.201698 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:59.232105 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9440357-a51c-412c-b66a-6ae295d1ca9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:59.233355 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9440357-a51c-412c-b66a-6ae295d1ca9b None admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:59.234067 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9440357-a51c-412c-b66a-6ae295d1ca9b None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:59.234534 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9440357-a51c-412c-b66a-6ae295d1ca9b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:59.236013 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9440357-a51c-412c-b66a-6ae295d1ca9b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:59.238283 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9440357-a51c-412c-b66a-6ae295d1ca9b None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:59.243173 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0a9feb-eedc-404f-8cfe-7c6b7984ff7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:59.243900 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0a9feb-eedc-404f-8cfe-7c6b7984ff7d None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:59.244248 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0a9feb-eedc-404f-8cfe-7c6b7984ff7d None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:59.244587 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0a9feb-eedc-404f-8cfe-7c6b7984ff7d None admin] PATH_INFO: `/v3/services` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:59.245284 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0a9feb-eedc-404f-8cfe-7c6b7984ff7d None admin] RBAC: Adding query filter params (name=network) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:59.245695 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0a9feb-eedc-404f-8cfe-7c6b7984ff7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:59.247793 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0a9feb-eedc-404f-8cfe-7c6b7984ff7d None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:59.253768 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 404/816] 10.222.0.61 () {56 vars in 1112 bytes} [Fri Aug 26 12:01:59 2022] GET /identity/v3/services?name=network => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:59.254406 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0a9feb-eedc-404f-8cfe-7c6b7984ff7d None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:59.267395 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 413/817] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:01:59 2022] POST /identity/v3/endpoints => generated 346 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:01:59.268227 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9440357-a51c-412c-b66a-6ae295d1ca9b None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:59.437539 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9440357-a51c-412c-b66a-6ae295d1ca9b None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:59.443480 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0a9feb-eedc-404f-8cfe-7c6b7984ff7d None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:59.479990 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3e8db9-ef2e-4ac9-a9ac-a7f51408105f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:59.480924 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e8db9-ef2e-4ac9-a9ac-a7f51408105f None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:59.481290 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e8db9-ef2e-4ac9-a9ac-a7f51408105f None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:59.482160 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e8db9-ef2e-4ac9-a9ac-a7f51408105f None admin] PATH_INFO: `/v3/services` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:59.482806 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3e8db9-ef2e-4ac9-a9ac-a7f51408105f None admin] RBAC: Adding query filter params (type=network) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:59.484090 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3e8db9-ef2e-4ac9-a9ac-a7f51408105f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:59.485295 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3e8db9-ef2e-4ac9-a9ac-a7f51408105f None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:59.493370 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 414/818] 10.222.0.61 () {56 vars in 1112 bytes} [Fri Aug 26 12:01:59 2022] GET /identity/v3/services?type=network => generated 350 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:59.494290 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3e8db9-ef2e-4ac9-a9ac-a7f51408105f None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:59.496031 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5a2929-6397-4959-9b5b-f360727d0315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:59.496691 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5a2929-6397-4959-9b5b-f360727d0315 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:59.497183 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5a2929-6397-4959-9b5b-f360727d0315 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:59.497628 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5a2929-6397-4959-9b5b-f360727d0315 None admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:59.498498 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5a2929-6397-4959-9b5b-f360727d0315 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:59.498807 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5a2929-6397-4959-9b5b-f360727d0315 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:59.500274 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5a2929-6397-4959-9b5b-f360727d0315 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:59.520336 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 405/819] 10.222.0.61 () {56 vars in 1148 bytes} [Fri Aug 26 12:01:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:59.520336 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5a2929-6397-4959-9b5b-f360727d0315 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:59.537961 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:59.553782 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:59.635687 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3e8db9-ef2e-4ac9-a9ac-a7f51408105f None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:59.639718 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5a2929-6397-4959-9b5b-f360727d0315 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:59.680320 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1db777-0921-4bb8-8af3-61c6d973b877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:59.681032 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1db777-0921-4bb8-8af3-61c6d973b877 None admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:59.681382 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1db777-0921-4bb8-8af3-61c6d973b877 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:59.681723 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1db777-0921-4bb8-8af3-61c6d973b877 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:59.682457 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1db777-0921-4bb8-8af3-61c6d973b877 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:59.682973 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a61ee5-ceed-474a-9910-64d4b683ebd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:59.683821 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a61ee5-ceed-474a-9910-64d4b683ebd2 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:59.684052 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1db777-0921-4bb8-8af3-61c6d973b877 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:59.685305 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a61ee5-ceed-474a-9910-64d4b683ebd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:59.685776 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a61ee5-ceed-474a-9910-64d4b683ebd2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:59.697433 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a61ee5-ceed-474a-9910-64d4b683ebd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:01:59.703955 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-c7a61ee5-ceed-474a-9910-64d4b683ebd2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 12:01:59.703955 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:01:59.703955 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:01:59.703955 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:01:59.703955 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:01:59.703955 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:01:59.703955 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:01:59.703955 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:01:59.703955 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:01:59.703955 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:01:59.703955 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:01:59.703955 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:01:59.703955 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:01:59.703955 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:01:59.703955 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:01:59.703955 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:01:59.704945 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:01:59.704945 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:01:59.704945 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:01:59.704945 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:01:59.704945 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:01:59.704945 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:01:59.704945 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:01:59.704945 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:01:59.704945 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:01:59.704945 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:01:59.704945 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:01:59.704945 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:01:59.704945 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:01:59.704945 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:01:59.704945 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:01:59.704945 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:01:59.705785 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:01:59.705785 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:01:59.705785 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:01:59.705785 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:01:59.705785 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:01:59.705785 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:01:59.705785 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:01:59.705785 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:01:59.705785 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:01:59.705785 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:01:59.705785 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 12:01:59.705785 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:01:59.708437 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 415/820] 10.222.0.61 () {56 vars in 1102 bytes} [Fri Aug 26 12:01:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:01:59.709451 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a61ee5-ceed-474a-9910-64d4b683ebd2 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:59.718668 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 406/821] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:01:59 2022] POST /identity/v3/endpoints => generated 337 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:01:59.719687 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1db777-0921-4bb8-8af3-61c6d973b877 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:01:59.831984 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:59.843833 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:01:59.856482 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a61ee5-ceed-474a-9910-64d4b683ebd2 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:59.906105 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1db777-0921-4bb8-8af3-61c6d973b877 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:01:59.926955 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06a770b-8247-465f-b183-3525e89715b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:59.927739 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06a770b-8247-465f-b183-3525e89715b7 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:59.928118 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06a770b-8247-465f-b183-3525e89715b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:59.928489 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06a770b-8247-465f-b183-3525e89715b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:59.929422 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06a770b-8247-465f-b183-3525e89715b7 None admin] RBAC: Adding query filter params (role.id=70ec1ba484714c8ca3adb8fe5845107c, scope.project.id=ecd68423a83a427681d44dbf266fce6e, user.id=7aacd7d6d17e4f10ba5c9107a12beafa) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:59.929918 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06a770b-8247-465f-b183-3525e89715b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:59.931536 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06a770b-8247-465f-b183-3525e89715b7 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:59.939123 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 416/822] 10.222.0.61 () {56 vars in 1374 bytes} [Fri Aug 26 12:01:59 2022] GET /identity/v3/role_assignments?user.id=7aacd7d6d17e4f10ba5c9107a12beafa&scope.project.id=ecd68423a83a427681d44dbf266fce6e&role.id=70ec1ba484714c8ca3adb8fe5845107c => generated 608 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:59.960717 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438f3184-e608-45c2-9228-4aebb9285737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:01:59.961463 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438f3184-e608-45c2-9228-4aebb9285737 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:01:59.961850 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438f3184-e608-45c2-9228-4aebb9285737 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:01:59.962221 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438f3184-e608-45c2-9228-4aebb9285737 None admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:01:59.963022 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438f3184-e608-45c2-9228-4aebb9285737 None admin] RBAC: Adding query filter params (name=service) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:01:59.963476 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438f3184-e608-45c2-9228-4aebb9285737 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:01:59.965018 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438f3184-e608-45c2-9228-4aebb9285737 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:01:59.978347 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 407/823] 10.222.0.61 () {56 vars in 1103 bytes} [Fri Aug 26 12:01:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:01:59.984513 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06a770b-8247-465f-b183-3525e89715b7 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:00.117677 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06a770b-8247-465f-b183-3525e89715b7 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:00.123771 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2aec86-6e5e-4dd9-a355-8025c9e7e9dd None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:00.124600 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2aec86-6e5e-4dd9-a355-8025c9e7e9dd None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:00.125018 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2aec86-6e5e-4dd9-a355-8025c9e7e9dd None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:00.127968 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 408/824] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:02:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:02:00.134753 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea009ba3-cbc0-4643-b0f7-07a9e1923585 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:00.135146 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea009ba3-cbc0-4643-b0f7-07a9e1923585 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:00.135374 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea009ba3-cbc0-4643-b0f7-07a9e1923585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:00.160457 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af6817e-c566-47c3-a5e7-b362e9915e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:00.161096 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af6817e-c566-47c3-a5e7-b362e9915e8b None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:00.161368 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af6817e-c566-47c3-a5e7-b362e9915e8b None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:00.161690 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af6817e-c566-47c3-a5e7-b362e9915e8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:00.164754 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-ea009ba3-cbc0-4643-b0f7-07a9e1923585 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:02:00.166114 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-ea009ba3-cbc0-4643-b0f7-07a9e1923585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:02:00.180729 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af6817e-c566-47c3-a5e7-b362e9915e8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:00.182175 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af6817e-c566-47c3-a5e7-b362e9915e8b None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:00.201523 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-3af6817e-c566-47c3-a5e7-b362e9915e8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:02:00.201523 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:00.201523 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:00.201523 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:00.201523 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:00.201523 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:00.201523 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:00.201523 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:00.201523 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:00.201523 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:00.201523 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:00.201523 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:00.201523 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:02:00.201523 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:02:00.201523 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:02:00.201523 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:02:00.202561 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:00.202561 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:00.202561 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:00.202561 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:00.202561 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:00.202561 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:00.202561 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:00.202561 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:00.202561 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:00.202561 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:00.202561 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:00.202561 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:00.202561 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:00.202561 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:00.202561 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:00.202561 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:00.202561 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:02:00.203909 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:02:00.203909 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:02:00.203909 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:00.203909 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 417/825] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:01:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:02:00.208224 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af6817e-c566-47c3-a5e7-b362e9915e8b None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:00.225742 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:00.263352 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 409/826] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:02:00 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 12:02:00.322851 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af6817e-c566-47c3-a5e7-b362e9915e8b None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:00.362295 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db49324-b287-4462-95fc-ed8c7107ce11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:00.362899 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db49324-b287-4462-95fc-ed8c7107ce11 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:00.363175 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db49324-b287-4462-95fc-ed8c7107ce11 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:00.363476 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db49324-b287-4462-95fc-ed8c7107ce11 None admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:00.364121 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db49324-b287-4462-95fc-ed8c7107ce11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:00.364418 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db49324-b287-4462-95fc-ed8c7107ce11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:00.365780 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db49324-b287-4462-95fc-ed8c7107ce11 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:00.378318 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 418/827] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:02:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:02:00.383997 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea009ba3-cbc0-4643-b0f7-07a9e1923585 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:00.445606 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:00.484820 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea009ba3-cbc0-4643-b0f7-07a9e1923585 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:00.519498 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a02842-e90e-40d2-8c01-8b94f7acf481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:00.520127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a02842-e90e-40d2-8c01-8b94f7acf481 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:00.520401 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a02842-e90e-40d2-8c01-8b94f7acf481 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:00.520670 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a02842-e90e-40d2-8c01-8b94f7acf481 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:00.527888 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a02842-e90e-40d2-8c01-8b94f7acf481 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:00.529919 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a02842-e90e-40d2-8c01-8b94f7acf481 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:00.541725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-c6a02842-e90e-40d2-8c01-8b94f7acf481 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 12:02:00.541725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:00.541725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:00.541725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:00.541725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:00.541725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:00.541725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:00.541725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:00.541725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:00.541725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:00.541725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:00.541725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:00.541725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 12:02:00.541725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 12:02:00.541725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 12:02:00.541725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 12:02:00.542709 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:00.542709 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:00.542709 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 12:02:00.542709 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:02:00.542709 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 12:02:00.542709 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:02:00.542709 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:00.542709 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:00.542709 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:00.542709 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:00.542709 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:00.542709 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:00.542709 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:00.542709 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:00.542709 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:00.542709 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:00.542709 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:00.542709 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:00.545262 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:00.545262 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:00.545262 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 12:02:00.545262 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 12:02:00.545262 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 12:02:00.545262 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 12:02:00.545262 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 12:02:00.545262 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 12:02:00.545262 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 12:02:00.545262 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:02:00.545262 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 410/828] 10.222.0.61 () {56 vars in 1117 bytes} [Fri Aug 26 12:02:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:02:00.546917 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db49324-b287-4462-95fc-ed8c7107ce11 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:00.549265 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a02842-e90e-40d2-8c01-8b94f7acf481 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:00.633321 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:00.675087 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a02842-e90e-40d2-8c01-8b94f7acf481 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:00.679237 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db49324-b287-4462-95fc-ed8c7107ce11 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:00.698424 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:00.712433 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1015252c-8ae9-41c7-a6af-aea7ea12d052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:00.713291 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1015252c-8ae9-41c7-a6af-aea7ea12d052 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:00.713575 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1015252c-8ae9-41c7-a6af-aea7ea12d052 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:00.713849 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1015252c-8ae9-41c7-a6af-aea7ea12d052 None admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:00.714578 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1015252c-8ae9-41c7-a6af-aea7ea12d052 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:00.715029 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1015252c-8ae9-41c7-a6af-aea7ea12d052 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:00.717385 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1015252c-8ae9-41c7-a6af-aea7ea12d052 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:00.725929 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02742ac4-b4ef-4e0f-ac75-dc711f99b8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:00.726826 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02742ac4-b4ef-4e0f-ac75-dc711f99b8b2 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:00.727576 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02742ac4-b4ef-4e0f-ac75-dc711f99b8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:00.728998 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02742ac4-b4ef-4e0f-ac75-dc711f99b8b2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:00.742559 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 411/829] 10.222.0.61 () {56 vars in 1149 bytes} [Fri Aug 26 12:02:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:00.746179 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1015252c-8ae9-41c7-a6af-aea7ea12d052 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:00.752520 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02742ac4-b4ef-4e0f-ac75-dc711f99b8b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:02:00.758719 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-02742ac4-b4ef-4e0f-ac75-dc711f99b8b2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 12:02:00.758719 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:00.758719 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:00.758719 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:00.758719 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:00.758719 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:00.758719 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:00.758719 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:00.758719 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:00.758719 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:00.758719 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:00.758719 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:00.758719 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:02:00.758719 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:02:00.758719 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:02:00.758719 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:02:00.759681 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:02:00.759681 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:02:00.759681 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:02:00.759681 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:02:00.759681 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:00.759681 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:00.759681 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:00.759681 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:00.759681 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:00.759681 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:00.759681 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:00.759681 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:00.759681 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:00.759681 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:00.759681 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:00.759681 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:00.760522 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:00.760522 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:00.760522 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:00.760522 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:00.760522 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:02:00.760522 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:02:00.760522 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:02:00.760522 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:02:00.760522 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:02:00.760522 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:02:00.760522 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 12:02:00.760522 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:00.763322 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 419/830] 10.222.0.61 () {56 vars in 1096 bytes} [Fri Aug 26 12:02:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 12:02:00.767504 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02742ac4-b4ef-4e0f-ac75-dc711f99b8b2 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:00.872825 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1015252c-8ae9-41c7-a6af-aea7ea12d052 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:00.889884 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02742ac4-b4ef-4e0f-ac75-dc711f99b8b2 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:00.917186 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29a7184-317d-43bb-aee5-12dae7c4d69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:00.917900 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29a7184-317d-43bb-aee5-12dae7c4d69c None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:00.918302 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29a7184-317d-43bb-aee5-12dae7c4d69c None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:00.919108 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29a7184-317d-43bb-aee5-12dae7c4d69c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:00.923029 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:00.929853 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf579503-0130-4849-a887-69425b5bed7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:00.930889 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf579503-0130-4849-a887-69425b5bed7a None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:00.931558 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf579503-0130-4849-a887-69425b5bed7a None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:00.932355 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf579503-0130-4849-a887-69425b5bed7a None admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:00.933180 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29a7184-317d-43bb-aee5-12dae7c4d69c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:00.933417 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf579503-0130-4849-a887-69425b5bed7a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:00.933839 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf579503-0130-4849-a887-69425b5bed7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:00.935868 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29a7184-317d-43bb-aee5-12dae7c4d69c None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:00.936099 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf579503-0130-4849-a887-69425b5bed7a None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:00.947248 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 420/831] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:02:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:00.952925 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf579503-0130-4849-a887-69425b5bed7a None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:00.957440 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-f29a7184-317d-43bb-aee5-12dae7c4d69c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:02:00.957440 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:00.957440 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:00.957440 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:00.957440 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:00.957440 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:00.957440 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:00.957440 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:00.957440 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:00.957440 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:00.957440 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:00.957440 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:00.957440 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:02:00.957440 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:02:00.957440 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:02:00.957440 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:02:00.958458 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:00.958458 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:00.958458 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:00.958458 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:00.958458 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:00.958458 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:00.958458 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:00.958458 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:00.958458 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:00.958458 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:00.958458 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:00.958458 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:00.958458 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:00.958458 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:00.958458 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:00.958458 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:00.958458 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:02:00.959468 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:02:00.959468 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:02:00.959468 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:02:00.963293 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 412/832] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:02:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:02:00.964527 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29a7184-317d-43bb-aee5-12dae7c4d69c None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:01.001594 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:01.073041 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf579503-0130-4849-a887-69425b5bed7a None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:01.098459 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29a7184-317d-43bb-aee5-12dae7c4d69c None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:01.113789 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1a82f2-9ab4-4337-bede-4a5c77873a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:01.115224 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a82f2-9ab4-4337-bede-4a5c77873a66 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:01.115985 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a82f2-9ab4-4337-bede-4a5c77873a66 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:01.116492 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a82f2-9ab4-4337-bede-4a5c77873a66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:01.132840 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1a82f2-9ab4-4337-bede-4a5c77873a66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:01.135150 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1a82f2-9ab4-4337-bede-4a5c77873a66 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:01.138182 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1601406-275d-4d5e-b4dc-50459b84325d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:01.138910 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1601406-275d-4d5e-b4dc-50459b84325d None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:01.139529 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1601406-275d-4d5e-b4dc-50459b84325d None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:01.139652 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1601406-275d-4d5e-b4dc-50459b84325d None admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:01.140313 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1601406-275d-4d5e-b4dc-50459b84325d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:01.140624 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1601406-275d-4d5e-b4dc-50459b84325d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:01.141990 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1601406-275d-4d5e-b4dc-50459b84325d None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:01.155020 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-9d1a82f2-9ab4-4337-bede-4a5c77873a66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:02:01.155020 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:01.155020 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:01.155020 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:01.155020 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:01.155020 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:01.155020 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:01.155020 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:01.155020 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:01.155020 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:01.155020 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:01.155020 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:01.155020 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:02:01.155020 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:02:01.155020 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:02:01.155020 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:02:01.156060 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:01.156060 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:01.156060 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:01.156060 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:01.156060 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:01.156060 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:01.156060 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:01.156060 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:01.156060 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:01.156060 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:01.156060 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:01.156060 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:01.156060 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:01.156060 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:01.156060 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:01.156060 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:01.156060 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:02:01.156950 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:02:01.156950 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:02:01.156950 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:01.159008 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 421/833] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:02:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:02:01.159339 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 413/834] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:02:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:01.164522 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1a82f2-9ab4-4337-bede-4a5c77873a66 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:01.166396 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1601406-275d-4d5e-b4dc-50459b84325d None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:01.226291 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:01.281277 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:01.286401 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1a82f2-9ab4-4337-bede-4a5c77873a66 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:01.291991 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1601406-275d-4d5e-b4dc-50459b84325d None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:01.328699 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb4168f-22e5-4038-8e73-f93d39cd63b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:01.329322 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb4168f-22e5-4038-8e73-f93d39cd63b2 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:01.329568 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb4168f-22e5-4038-8e73-f93d39cd63b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:01.329839 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb4168f-22e5-4038-8e73-f93d39cd63b2 None admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:01.330488 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb4168f-22e5-4038-8e73-f93d39cd63b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:01.330781 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb4168f-22e5-4038-8e73-f93d39cd63b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:01.331556 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f9ffc1-4aba-4f33-8936-748e2f9c1955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:01.332159 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb4168f-22e5-4038-8e73-f93d39cd63b2 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:01.332624 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f9ffc1-4aba-4f33-8936-748e2f9c1955 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:01.333107 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f9ffc1-4aba-4f33-8936-748e2f9c1955 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:01.333661 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f9ffc1-4aba-4f33-8936-748e2f9c1955 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:01.341460 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f9ffc1-4aba-4f33-8936-748e2f9c1955 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:01.343415 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f9ffc1-4aba-4f33-8936-748e2f9c1955 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:01.349814 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 422/835] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:02:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:02:01.351667 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb4168f-22e5-4038-8e73-f93d39cd63b2 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:01.358682 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-39f9ffc1-4aba-4f33-8936-748e2f9c1955 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:02:01.358682 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:01.358682 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:01.358682 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:01.358682 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:01.358682 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:01.358682 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:01.358682 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:01.358682 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:01.358682 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:01.358682 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:01.358682 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:01.358682 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 12:02:01.358682 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 12:02:01.358682 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 12:02:01.359848 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 12:02:01.359848 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:01.359848 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:01.359848 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:01.359848 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:01.359848 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:01.359848 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:01.359848 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:01.359848 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:01.359848 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:01.359848 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:01.359848 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:01.359848 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:01.359848 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:01.359848 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:01.359848 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:01.359848 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:01.364226 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 12:02:01.364226 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 12:02:01.364226 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 12:02:01.364226 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 12:02:01.364226 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 12:02:01.364226 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 12:02:01.364226 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:02:01.364226 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:02:01.364226 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 414/836] 10.222.0.61 () {56 vars in 1111 bytes} [Fri Aug 26 12:02:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:02:01.367516 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f9ffc1-4aba-4f33-8936-748e2f9c1955 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:01.480283 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb4168f-22e5-4038-8e73-f93d39cd63b2 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:01.483530 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:01.488130 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f9ffc1-4aba-4f33-8936-748e2f9c1955 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:01.521212 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6ae2f1-15be-4340-8962-b6e1b9443782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:01.521840 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6ae2f1-15be-4340-8962-b6e1b9443782 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:01.522116 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6ae2f1-15be-4340-8962-b6e1b9443782 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:01.522383 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6ae2f1-15be-4340-8962-b6e1b9443782 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:01.524174 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d04b2b-5dca-478a-a0b2-0aaaf724ea3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:01.524746 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d04b2b-5dca-478a-a0b2-0aaaf724ea3e None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:01.525029 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d04b2b-5dca-478a-a0b2-0aaaf724ea3e None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:01.525307 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d04b2b-5dca-478a-a0b2-0aaaf724ea3e None admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:01.525998 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d04b2b-5dca-478a-a0b2-0aaaf724ea3e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:01.526303 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d04b2b-5dca-478a-a0b2-0aaaf724ea3e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:01.527786 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d04b2b-5dca-478a-a0b2-0aaaf724ea3e None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:01.531938 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6ae2f1-15be-4340-8962-b6e1b9443782 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:01.533575 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6ae2f1-15be-4340-8962-b6e1b9443782 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:01.540958 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 415/837] 10.222.0.61 () {56 vars in 1148 bytes} [Fri Aug 26 12:02:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:02:01.542174 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e5b503-4c38-4c89-a50b-5f9b9309c6c3 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:01.542457 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e5b503-4c38-4c89-a50b-5f9b9309c6c3 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:01.542728 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e5b503-4c38-4c89-a50b-5f9b9309c6c3 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:01.544795 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 416/838] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:02:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:02:01.548924 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e5b503-4c38-4c89-a50b-5f9b9309c6c3 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:01.553554 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-3c6ae2f1-15be-4340-8962-b6e1b9443782 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 26 12:02:01.553554 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:01.553554 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:01.553554 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:01.553554 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:01.553554 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:01.553554 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:01.553554 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:01.553554 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:01.553554 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:01.553554 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:01.553554 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:01.553554 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 12:02:01.553554 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 12:02:01.553554 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 12:02:01.553554 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 12:02:01.554540 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:01.554540 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:01.554540 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 12:02:01.554540 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:02:01.554540 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 12:02:01.554540 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:02:01.554540 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:01.554540 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:01.554540 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:01.554540 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:01.554540 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:01.554540 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:01.554540 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:01.554540 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:01.554540 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:01.554540 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:01.554540 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:01.554540 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:01.555563 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:01.555563 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:01.555563 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 12:02:01.555563 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 12:02:01.555563 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 12:02:01.555563 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 12:02:01.555563 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 12:02:01.555563 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 12:02:01.555563 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 26 12:02:01.555563 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:01.555563 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 423/839] 10.222.0.61 () {56 vars in 1096 bytes} [Fri Aug 26 12:02:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 12:02:01.556193 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31074270-62ae-4250-a1a7-3cd66df2f02b None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:01.556792 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31074270-62ae-4250-a1a7-3cd66df2f02b None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:01.556792 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31074270-62ae-4250-a1a7-3cd66df2f02b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:01.569007 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:01.577076 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-31074270-62ae-4250-a1a7-3cd66df2f02b None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:02:01.578748 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-31074270-62ae-4250-a1a7-3cd66df2f02b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:02:01.656827 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 424/840] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:02:01 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 12:02:01.660727 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31074270-62ae-4250-a1a7-3cd66df2f02b None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:01.664346 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e5b503-4c38-4c89-a50b-5f9b9309c6c3 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:01.709985 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ea9c59-ef37-4e2c-b056-39cbbe672748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:01.710920 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea9c59-ef37-4e2c-b056-39cbbe672748 None admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:01.711406 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea9c59-ef37-4e2c-b056-39cbbe672748 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:01.711901 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea9c59-ef37-4e2c-b056-39cbbe672748 None admin] PATH_INFO: `/v3/projects/ecd68423a83a427681d44dbf266fce6e/users/755b04cae45746569bfbf92f2f213a58/roles/519347d2bd2c4dc1aadb0b9f8c3c8e3d` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:01.728021 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ea9c59-ef37-4e2c-b056-39cbbe672748 None admin] RBAC: Authorizing `identity:create_grant(project_id=ecd68423a83a427681d44dbf266fce6e, user_id=755b04cae45746569bfbf92f2f213a58, role_id=519347d2bd2c4dc1aadb0b9f8c3c8e3d)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:01.729957 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ea9c59-ef37-4e2c-b056-39cbbe672748 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:01.759266 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 417/841] 10.222.0.61 () {58 vars in 1439 bytes} [Fri Aug 26 12:02:01 2022] PUT /identity/v3/projects/ecd68423a83a427681d44dbf266fce6e/users/755b04cae45746569bfbf92f2f213a58/roles/519347d2bd2c4dc1aadb0b9f8c3c8e3d => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:02:01.782927 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:01.790716 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31074270-62ae-4250-a1a7-3cd66df2f02b None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:01.830192 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1777db32-b0e5-43de-a520-aa55ec147cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:01.831255 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1777db32-b0e5-43de-a520-aa55ec147cc5 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:01.831740 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1777db32-b0e5-43de-a520-aa55ec147cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:01.832209 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1777db32-b0e5-43de-a520-aa55ec147cc5 None admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:01.833176 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1777db32-b0e5-43de-a520-aa55ec147cc5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:01.833711 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1777db32-b0e5-43de-a520-aa55ec147cc5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:01.835392 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1777db32-b0e5-43de-a520-aa55ec147cc5 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:01.857020 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 425/842] 10.222.0.61 () {56 vars in 1135 bytes} [Fri Aug 26 12:02:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:01.863690 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ea9c59-ef37-4e2c-b056-39cbbe672748 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:01.935037 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1777db32-b0e5-43de-a520-aa55ec147cc5 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:01.936769 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:01.986369 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ea9c59-ef37-4e2c-b056-39cbbe672748 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:02.029629 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968755b0-2e66-4360-bc13-1d68c65cb3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:02.030500 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968755b0-2e66-4360-bc13-1d68c65cb3ca None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:02.030779 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968755b0-2e66-4360-bc13-1d68c65cb3ca None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:02.031049 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968755b0-2e66-4360-bc13-1d68c65cb3ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:02.073931 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1777db32-b0e5-43de-a520-aa55ec147cc5 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:02.109911 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d34edb-b0cb-49b1-9ce5-2b72d44fab0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:02.110541 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d34edb-b0cb-49b1-9ce5-2b72d44fab0e None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:02.110820 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d34edb-b0cb-49b1-9ce5-2b72d44fab0e None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:02.110938 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968755b0-2e66-4360-bc13-1d68c65cb3ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:02.111107 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d34edb-b0cb-49b1-9ce5-2b72d44fab0e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:02.111694 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d34edb-b0cb-49b1-9ce5-2b72d44fab0e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:02.112419 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968755b0-2e66-4360-bc13-1d68c65cb3ca None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:02.113037 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d34edb-b0cb-49b1-9ce5-2b72d44fab0e None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:02.122816 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-86d34edb-b0cb-49b1-9ce5-2b72d44fab0e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 12:02:02.122816 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:02.122816 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 12:02:02.122816 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 12:02:02.122816 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 12:02:02.122816 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 12:02:02.122816 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 12:02:02.122816 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 12:02:02.122816 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 12:02:02.122816 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:02.122816 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 12:02:02.122816 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:02.122816 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:02.122816 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:02.122816 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:02.122816 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:02.122816 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:02.123927 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:02.123927 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:02.123927 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:02.123927 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:02.123927 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:02.123927 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:02.123927 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 12:02:02.123927 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 12:02:02.123927 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 12:02:02.123927 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 12:02:02.123927 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:02.123927 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:02.123927 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:02.123927 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:02.123927 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:02.123927 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:02.124789 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:02.124789 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:02.124789 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:02.124789 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:02.124789 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:02.124789 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:02.124789 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:02.124789 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:02.124789 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:02.124789 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:02.124789 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 12:02:02.124789 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 12:02:02.124789 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 12:02:02.124789 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:02.124789 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 426/843] 10.222.0.61 () {56 vars in 1114 bytes} [Fri Aug 26 12:02:01 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:02:02.125628 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031162ca-3afd-4031-8aaa-78ec5c13c159 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:02.125628 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031162ca-3afd-4031-8aaa-78ec5c13c159 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:02.125758 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031162ca-3afd-4031-8aaa-78ec5c13c159 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:02.131121 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 427/844] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:02:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:02:02.131952 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031162ca-3afd-4031-8aaa-78ec5c13c159 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:02.153720 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:02.186544 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:02.230051 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 418/845] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:02:01 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 12:02:02.231709 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c60f437-c65b-433d-86e9-d1d9c57fc7e8 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:02.231993 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c60f437-c65b-433d-86e9-d1d9c57fc7e8 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:02.232266 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c60f437-c65b-433d-86e9-d1d9c57fc7e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:02.248035 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031162ca-3afd-4031-8aaa-78ec5c13c159 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:02.250530 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-0c60f437-c65b-433d-86e9-d1d9c57fc7e8 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:02:02.251744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-0c60f437-c65b-433d-86e9-d1d9c57fc7e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:02:02.289809 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c581da8d-f7ae-4441-961a-f95193e8c033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:02.290412 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c581da8d-f7ae-4441-961a-f95193e8c033 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:02.290688 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c581da8d-f7ae-4441-961a-f95193e8c033 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:02.290959 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c581da8d-f7ae-4441-961a-f95193e8c033 None admin] PATH_INFO: `/v3/services` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:02.291615 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c581da8d-f7ae-4441-961a-f95193e8c033 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:02.291917 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c581da8d-f7ae-4441-961a-f95193e8c033 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:02.293268 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c581da8d-f7ae-4441-961a-f95193e8c033 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:02.303484 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 428/846] 10.222.0.61 () {56 vars in 1114 bytes} [Fri Aug 26 12:02:02 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:02.303484 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c581da8d-f7ae-4441-961a-f95193e8c033 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:02.323346 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 419/847] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:02:02 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 12:02:02.324516 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c60f437-c65b-433d-86e9-d1d9c57fc7e8 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:02.400861 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:02.421577 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c581da8d-f7ae-4441-961a-f95193e8c033 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:02.453422 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:02.456849 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6df291-3f73-4321-badb-dfb27bd32b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:02.457452 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6df291-3f73-4321-badb-dfb27bd32b2f None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:02.457835 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6df291-3f73-4321-badb-dfb27bd32b2f None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:02.458107 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6df291-3f73-4321-badb-dfb27bd32b2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:02.465637 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c60f437-c65b-433d-86e9-d1d9c57fc7e8 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:02.469869 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6df291-3f73-4321-badb-dfb27bd32b2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:02.471335 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6df291-3f73-4321-badb-dfb27bd32b2f None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:02.486655 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-bf6df291-3f73-4321-badb-dfb27bd32b2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:02:02.486655 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:02.486655 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:02.486655 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:02.486655 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:02.486655 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:02.486655 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:02.486655 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:02.486655 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:02.486655 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:02.486655 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:02.486655 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:02.486655 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:02:02.486655 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:02:02.486655 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:02:02.486655 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:02:02.487717 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:02.487717 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:02.487717 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:02.487717 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:02.487717 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:02.487717 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:02.487717 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:02.487717 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:02.487717 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:02.487717 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:02.487717 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:02.487717 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:02.487717 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:02.487717 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:02.487717 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:02.487717 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:02.487717 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:02:02.488629 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:02:02.488629 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:02:02.488629 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:02.488629 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 429/848] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:02:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:02:02.496118 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6df291-3f73-4321-badb-dfb27bd32b2f None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:02.514385 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae646ea4-be2a-460d-93e0-6e7d3ac48d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:02.514994 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae646ea4-be2a-460d-93e0-6e7d3ac48d5e None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:02.515323 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae646ea4-be2a-460d-93e0-6e7d3ac48d5e None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:02.515597 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae646ea4-be2a-460d-93e0-6e7d3ac48d5e None admin] PATH_INFO: `/v3/services` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:02.516525 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae646ea4-be2a-460d-93e0-6e7d3ac48d5e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:02.516829 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae646ea4-be2a-460d-93e0-6e7d3ac48d5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:02.518472 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae646ea4-be2a-460d-93e0-6e7d3ac48d5e None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:02.524413 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 420/849] 10.222.0.61 () {56 vars in 1114 bytes} [Fri Aug 26 12:02:02 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:02.618854 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae646ea4-be2a-460d-93e0-6e7d3ac48d5e None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:02.619423 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6df291-3f73-4321-badb-dfb27bd32b2f None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:02.659354 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:02.666577 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9654b0e-33fc-4e3e-b23e-7c241d2f32eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:02.667336 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9654b0e-33fc-4e3e-b23e-7c241d2f32eb None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:02.667734 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9654b0e-33fc-4e3e-b23e-7c241d2f32eb None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:02.668103 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9654b0e-33fc-4e3e-b23e-7c241d2f32eb None admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:02.670640 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9654b0e-33fc-4e3e-b23e-7c241d2f32eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:02.670779 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9654b0e-33fc-4e3e-b23e-7c241d2f32eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:02.672312 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9654b0e-33fc-4e3e-b23e-7c241d2f32eb None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:02.687798 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 430/850] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:02:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:02.694778 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9654b0e-33fc-4e3e-b23e-7c241d2f32eb None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:02.754580 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae646ea4-be2a-460d-93e0-6e7d3ac48d5e None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:02.798957 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1b9d59-9d5e-45c7-bf76-029467a30957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:02.799749 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1b9d59-9d5e-45c7-bf76-029467a30957 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:02.800101 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1b9d59-9d5e-45c7-bf76-029467a30957 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:02.800444 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1b9d59-9d5e-45c7-bf76-029467a30957 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:02.801099 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1b9d59-9d5e-45c7-bf76-029467a30957 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:02.802542 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1b9d59-9d5e-45c7-bf76-029467a30957 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:02.815343 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-de1b9d59-9d5e-45c7-bf76-029467a30957 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 12:02:02.815343 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:02.815343 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 12:02:02.815343 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 12:02:02.815343 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 12:02:02.815343 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 12:02:02.815343 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 12:02:02.815343 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 12:02:02.815343 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 12:02:02.815343 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:02:02.815343 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 12:02:02.815343 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:02:02.815343 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:02.815343 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:02.815343 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:02.815343 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:02.816318 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:02.816318 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:02.816318 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:02.816318 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:02.816318 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:02.816318 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:02.816318 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:02.816318 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 12:02:02.816318 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 12:02:02.816318 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 12:02:02.816318 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 12:02:02.816318 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:02.816318 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:02.816318 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:02.816318 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:02.816318 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:02.816318 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:02.817245 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:02.817245 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:02.817245 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:02.817245 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:02.817245 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:02.817245 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:02.817245 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:02.817245 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:02.817245 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:02.817245 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:02.817245 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 12:02:02.817245 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 12:02:02.817245 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 12:02:02.817245 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:02:02.817245 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 421/851] 10.222.0.61 () {56 vars in 1132 bytes} [Fri Aug 26 12:02:02 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:02:02.823402 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1b9d59-9d5e-45c7-bf76-029467a30957 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:02.837126 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9654b0e-33fc-4e3e-b23e-7c241d2f32eb None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:02.852016 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:02.889566 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7022b7f2-58b4-4b3e-afe6-ab82af20a641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:02.890739 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7022b7f2-58b4-4b3e-afe6-ab82af20a641 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:02.891676 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7022b7f2-58b4-4b3e-afe6-ab82af20a641 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:02.892125 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7022b7f2-58b4-4b3e-afe6-ab82af20a641 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:02.901443 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7022b7f2-58b4-4b3e-afe6-ab82af20a641 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:02.903880 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7022b7f2-58b4-4b3e-afe6-ab82af20a641 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:02.919350 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-7022b7f2-58b4-4b3e-afe6-ab82af20a641 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:02:02.919350 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:02.919350 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:02.919350 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:02.919350 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:02.919350 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:02.919350 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:02.919350 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:02.919350 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:02.919350 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:02.919350 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:02.919350 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:02.919350 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 12:02:02.919350 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 12:02:02.919350 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 12:02:02.920324 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 12:02:02.920324 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:02.920324 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:02.920324 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:02.920324 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:02.920324 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:02.920324 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:02.920324 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:02.920324 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:02.920324 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:02.920324 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:02.920324 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:02.920324 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:02.920324 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:02.920324 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:02.920324 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:02.920324 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:02.921209 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 12:02:02.921209 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 12:02:02.921209 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 12:02:02.921209 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 12:02:02.921209 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 12:02:02.921209 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 12:02:02.921209 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:02:02.921209 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:02.925388 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 431/852] 10.222.0.61 () {56 vars in 1111 bytes} [Fri Aug 26 12:02:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:02:02.926307 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7022b7f2-58b4-4b3e-afe6-ab82af20a641 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:02.957262 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1b9d59-9d5e-45c7-bf76-029467a30957 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:02.982657 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:02.997953 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bc67fc-3db3-4e9b-a8be-6fe726ab134b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:02.999289 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bc67fc-3db3-4e9b-a8be-6fe726ab134b None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:03.000330 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bc67fc-3db3-4e9b-a8be-6fe726ab134b None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:03.000908 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bc67fc-3db3-4e9b-a8be-6fe726ab134b None admin] PATH_INFO: `/v3/services` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:03.001737 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bc67fc-3db3-4e9b-a8be-6fe726ab134b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:03.002237 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bc67fc-3db3-4e9b-a8be-6fe726ab134b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:03.003983 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bc67fc-3db3-4e9b-a8be-6fe726ab134b None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:03.009960 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 422/853] 10.222.0.61 () {56 vars in 1126 bytes} [Fri Aug 26 12:02:02 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:02:03.016068 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bc67fc-3db3-4e9b-a8be-6fe726ab134b None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:03.044929 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7022b7f2-58b4-4b3e-afe6-ab82af20a641 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:03.091043 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606c3834-4da9-4cd4-8877-ff69cec611ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:03.091728 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606c3834-4da9-4cd4-8877-ff69cec611ed None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:03.092010 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606c3834-4da9-4cd4-8877-ff69cec611ed None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:03.092291 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606c3834-4da9-4cd4-8877-ff69cec611ed None admin] PATH_INFO: `/v3/projects` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:03.093231 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606c3834-4da9-4cd4-8877-ff69cec611ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:03.093550 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606c3834-4da9-4cd4-8877-ff69cec611ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:03.095710 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606c3834-4da9-4cd4-8877-ff69cec611ed None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:03.118624 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 432/854] 10.222.0.61 () {56 vars in 1148 bytes} [Fri Aug 26 12:02:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:02:03.120004 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606c3834-4da9-4cd4-8877-ff69cec611ed None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:03.129937 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:03.147976 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bc67fc-3db3-4e9b-a8be-6fe726ab134b None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:03.183616 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f415ed2-7101-4da8-b4eb-9b13e2b2a973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:03.184025 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f415ed2-7101-4da8-b4eb-9b13e2b2a973 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:03.184881 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f415ed2-7101-4da8-b4eb-9b13e2b2a973 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:03.185151 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f415ed2-7101-4da8-b4eb-9b13e2b2a973 None admin] PATH_INFO: `/v3/services` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:03.185784 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f415ed2-7101-4da8-b4eb-9b13e2b2a973 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:03.186095 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f415ed2-7101-4da8-b4eb-9b13e2b2a973 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:03.187547 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f415ed2-7101-4da8-b4eb-9b13e2b2a973 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:03.199251 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 423/855] 10.222.0.61 () {56 vars in 1126 bytes} [Fri Aug 26 12:02:03 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:03.246357 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606c3834-4da9-4cd4-8877-ff69cec611ed None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:03.262096 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:03.288197 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be67617f-21b1-4654-a210-31d2b0a34c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:03.288811 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be67617f-21b1-4654-a210-31d2b0a34c41 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:03.289091 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be67617f-21b1-4654-a210-31d2b0a34c41 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:03.289360 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be67617f-21b1-4654-a210-31d2b0a34c41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:03.290093 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be67617f-21b1-4654-a210-31d2b0a34c41 None admin] RBAC: Adding query filter params (role.id=70ec1ba484714c8ca3adb8fe5845107c, scope.project.id=ecd68423a83a427681d44dbf266fce6e, user.id=ead95a244d11432d9fe9f718e1a75f38) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:03.290466 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be67617f-21b1-4654-a210-31d2b0a34c41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:03.292029 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be67617f-21b1-4654-a210-31d2b0a34c41 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:03.299447 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 433/856] 10.222.0.61 () {56 vars in 1374 bytes} [Fri Aug 26 12:02:03 2022] GET /identity/v3/role_assignments?user.id=ead95a244d11432d9fe9f718e1a75f38&scope.project.id=ecd68423a83a427681d44dbf266fce6e&role.id=70ec1ba484714c8ca3adb8fe5845107c => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:03.760853 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ec9e3e-7eb1-4205-a848-034ec1e2adfc None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:03.761137 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ec9e3e-7eb1-4205-a848-034ec1e2adfc None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:03.761413 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ec9e3e-7eb1-4205-a848-034ec1e2adfc None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:03.763574 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 424/857] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:02:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:02:03.770096 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e01c04-33f3-4f30-bdc2-ca26a63da178 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:03.770552 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e01c04-33f3-4f30-bdc2-ca26a63da178 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:03.770857 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e01c04-33f3-4f30-bdc2-ca26a63da178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:03.790867 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-b6e01c04-33f3-4f30-bdc2-ca26a63da178 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:02:03.792636 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-b6e01c04-33f3-4f30-bdc2-ca26a63da178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:02:03.863952 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 434/858] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:02:03 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 12:02:04.138965 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ec9e3e-7eb1-4205-a848-034ec1e2adfc None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:04.248692 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ec9e3e-7eb1-4205-a848-034ec1e2adfc None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:04.284448 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a85e5c-34ce-4d69-8b0c-86f226f9cb5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:04.285077 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a85e5c-34ce-4d69-8b0c-86f226f9cb5e None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:04.285362 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a85e5c-34ce-4d69-8b0c-86f226f9cb5e None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:04.285637 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a85e5c-34ce-4d69-8b0c-86f226f9cb5e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:04.297021 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a85e5c-34ce-4d69-8b0c-86f226f9cb5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:02:04.301652 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-76a85e5c-34ce-4d69-8b0c-86f226f9cb5e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 12:02:04.301652 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:04.301652 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:04.301652 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:04.301652 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:04.301652 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:04.301652 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:04.301652 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:04.301652 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:04.301652 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:04.301652 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:04.301652 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:04.301652 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:02:04.301652 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:02:04.301652 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:02:04.301652 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:02:04.302599 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:02:04.302599 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:02:04.302599 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:02:04.302599 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:02:04.302599 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:04.302599 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:04.302599 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:04.302599 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:04.302599 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:04.302599 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:04.302599 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:04.302599 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:04.302599 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:04.302599 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:04.302599 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:04.302599 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:04.303475 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:04.303475 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:04.303475 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:04.303475 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:04.303475 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:02:04.303475 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:02:04.303475 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:02:04.303475 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:02:04.303475 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:02:04.303475 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:02:04.303475 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 12:02:04.303475 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:02:04.303475 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 425/859] 10.222.0.61 () {56 vars in 1102 bytes} [Fri Aug 26 12:02:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:02:04.308834 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e01c04-33f3-4f30-bdc2-ca26a63da178 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:04.419322 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e01c04-33f3-4f30-bdc2-ca26a63da178 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:04.427241 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:04.454991 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d171c866-4a00-45ba-9f12-8c44c57131f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:04.455647 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171c866-4a00-45ba-9f12-8c44c57131f3 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:04.455977 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171c866-4a00-45ba-9f12-8c44c57131f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:04.456256 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d171c866-4a00-45ba-9f12-8c44c57131f3 None admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:04.456941 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d171c866-4a00-45ba-9f12-8c44c57131f3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:04.457296 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d171c866-4a00-45ba-9f12-8c44c57131f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:04.458764 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d171c866-4a00-45ba-9f12-8c44c57131f3 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:04.469604 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 435/860] 10.222.0.61 () {56 vars in 1103 bytes} [Fri Aug 26 12:02:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:04.474926 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a85e5c-34ce-4d69-8b0c-86f226f9cb5e None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:04.491587 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58001fb-b72a-473e-a25d-ea7be1d46e4e None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:04.491988 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:04.492121 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58001fb-b72a-473e-a25d-ea7be1d46e4e None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:04.492394 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58001fb-b72a-473e-a25d-ea7be1d46e4e None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:04.493686 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 436/861] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:02:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 12:02:04.499054 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d2296b-39ad-42c3-a940-6d40ce57a48b None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:04.499364 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d2296b-39ad-42c3-a940-6d40ce57a48b None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:04.499631 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d2296b-39ad-42c3-a940-6d40ce57a48b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:04.519713 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-89d2296b-39ad-42c3-a940-6d40ce57a48b None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:02:04.520893 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-89d2296b-39ad-42c3-a940-6d40ce57a48b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:02:04.601685 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a85e5c-34ce-4d69-8b0c-86f226f9cb5e None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:04.603349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 437/862] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:02:04 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 12:02:04.640726 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06457d32-9799-4785-a8c9-6b0f432029d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:04.641350 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06457d32-9799-4785-a8c9-6b0f432029d9 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:04.641626 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06457d32-9799-4785-a8c9-6b0f432029d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:04.641892 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06457d32-9799-4785-a8c9-6b0f432029d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:04.652693 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06457d32-9799-4785-a8c9-6b0f432029d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:04.655023 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06457d32-9799-4785-a8c9-6b0f432029d9 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:04.669711 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-06457d32-9799-4785-a8c9-6b0f432029d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:02:04.669711 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:04.669711 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:04.669711 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:04.669711 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:04.669711 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:04.669711 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:04.669711 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:04.669711 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:04.669711 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:04.669711 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:04.669711 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:04.669711 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:02:04.669711 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:02:04.669711 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:02:04.669711 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:02:04.670648 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:04.670648 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:04.670648 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:04.670648 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:04.670648 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:04.670648 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:04.670648 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:04.670648 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:04.670648 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:04.670648 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:04.670648 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:04.670648 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:04.670648 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:04.670648 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:04.670648 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:04.670648 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:04.670648 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:02:04.671577 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:02:04.671577 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:02:04.671577 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:02:04.675338 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 426/863] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:02:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:02:04.676998 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d2296b-39ad-42c3-a940-6d40ce57a48b None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:04.777535 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:04.783764 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d2296b-39ad-42c3-a940-6d40ce57a48b None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:04.819744 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ee4d2d-43c6-44db-9e49-4d3962a3876f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:04.820468 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ee4d2d-43c6-44db-9e49-4d3962a3876f None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:04.820747 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ee4d2d-43c6-44db-9e49-4d3962a3876f None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:04.821012 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ee4d2d-43c6-44db-9e49-4d3962a3876f None admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:04.821690 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ee4d2d-43c6-44db-9e49-4d3962a3876f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:04.821997 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ee4d2d-43c6-44db-9e49-4d3962a3876f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:04.823370 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ee4d2d-43c6-44db-9e49-4d3962a3876f None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:04.839333 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 438/864] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:02:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:04.841591 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06457d32-9799-4785-a8c9-6b0f432029d9 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:04.905314 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:04.945127 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ee4d2d-43c6-44db-9e49-4d3962a3876f None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:04.951227 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06457d32-9799-4785-a8c9-6b0f432029d9 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:04.988836 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c530e74-6df2-43a5-bddd-0148a5fea80a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:04.989470 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c530e74-6df2-43a5-bddd-0148a5fea80a None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:04.989747 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c530e74-6df2-43a5-bddd-0148a5fea80a None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:04.990039 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c530e74-6df2-43a5-bddd-0148a5fea80a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:04.998246 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c530e74-6df2-43a5-bddd-0148a5fea80a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:04.999832 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c530e74-6df2-43a5-bddd-0148a5fea80a None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:05.012004 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-2c530e74-6df2-43a5-bddd-0148a5fea80a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 12:02:05.012004 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:05.012004 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:05.012004 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:05.012004 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:05.012004 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:05.012004 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:05.012004 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:05.012004 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:05.012004 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:05.012004 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:05.012004 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:05.012004 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 12:02:05.012004 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 12:02:05.012004 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 12:02:05.012004 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 12:02:05.013040 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:05.013040 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:05.013040 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 12:02:05.013040 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:02:05.013040 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 12:02:05.013040 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:02:05.013040 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:05.013040 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:05.013040 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:05.013040 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:05.013040 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:05.013040 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:05.013040 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:05.013040 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:05.013040 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:05.013040 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:05.013040 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:05.013040 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:05.018885 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:05.018885 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:05.018885 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 12:02:05.018885 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 12:02:05.018885 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 12:02:05.018885 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 12:02:05.018885 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 12:02:05.018885 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 12:02:05.018885 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 12:02:05.018885 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:02:05.018885 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 427/865] 10.222.0.61 () {56 vars in 1117 bytes} [Fri Aug 26 12:02:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 12:02:05.021362 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c530e74-6df2-43a5-bddd-0148a5fea80a None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:05.076172 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ee4d2d-43c6-44db-9e49-4d3962a3876f None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:05.118569 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932e8893-6c92-408a-ac54-63a4d936e914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:05.119185 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e8893-6c92-408a-ac54-63a4d936e914 None admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:05.119488 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e8893-6c92-408a-ac54-63a4d936e914 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:05.119757 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e8893-6c92-408a-ac54-63a4d936e914 None admin] PATH_INFO: `/v3/services` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:05.120426 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932e8893-6c92-408a-ac54-63a4d936e914 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:05.122298 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932e8893-6c92-408a-ac54-63a4d936e914 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:05.147238 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 439/866] 10.222.0.61 () {60 vars in 1141 bytes} [Fri Aug 26 12:02:04 2022] POST /identity/v3/services => generated 251 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:02:05.157294 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c530e74-6df2-43a5-bddd-0148a5fea80a None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:05.183629 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:05.208280 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b5c7d2-1627-4ef0-b09a-1746adb929a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:05.209217 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b5c7d2-1627-4ef0-b09a-1746adb929a2 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:05.209680 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b5c7d2-1627-4ef0-b09a-1746adb929a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:05.210146 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b5c7d2-1627-4ef0-b09a-1746adb929a2 None admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:05.211083 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b5c7d2-1627-4ef0-b09a-1746adb929a2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:05.211605 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b5c7d2-1627-4ef0-b09a-1746adb929a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:05.213277 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b5c7d2-1627-4ef0-b09a-1746adb929a2 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:05.229321 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ed386-16c4-4c6a-9948-1dec62f086bf None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:05.229622 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ed386-16c4-4c6a-9948-1dec62f086bf None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:05.229893 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ed386-16c4-4c6a-9948-1dec62f086bf None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:05.236306 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 440/867] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:02:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 12:02:05.237737 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2da588-7eb7-4d71-b0fa-488ef084e5e7 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:05.238028 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2da588-7eb7-4d71-b0fa-488ef084e5e7 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:05.238309 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2da588-7eb7-4d71-b0fa-488ef084e5e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:05.239560 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 428/868] 10.222.0.61 () {56 vars in 1149 bytes} [Fri Aug 26 12:02:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:02:05.245237 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b5c7d2-1627-4ef0-b09a-1746adb929a2 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:05.265429 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-4a2da588-7eb7-4d71-b0fa-488ef084e5e7 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:02:05.267139 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-4a2da588-7eb7-4d71-b0fa-488ef084e5e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:02:05.288386 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:05.358468 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 441/869] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:02:05 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 12:02:05.363583 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec47b03e-bba0-4b1e-9839-2cdde657a423 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:05.363583 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec47b03e-bba0-4b1e-9839-2cdde657a423 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:05.363583 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec47b03e-bba0-4b1e-9839-2cdde657a423 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:05.363583 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 442/870] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:02:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:02:05.366786 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70fa44-4658-4695-9a93-3f7b625266ee None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:05.367071 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70fa44-4658-4695-9a93-3f7b625266ee None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:05.367439 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70fa44-4658-4695-9a93-3f7b625266ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:05.374381 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b5c7d2-1627-4ef0-b09a-1746adb929a2 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:05.385587 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-5e70fa44-4658-4695-9a93-3f7b625266ee None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:02:05.387052 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-5e70fa44-4658-4695-9a93-3f7b625266ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:02:05.410687 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5266378-c33e-4afd-8457-422ac0b7b026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:05.411324 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5266378-c33e-4afd-8457-422ac0b7b026 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:05.411608 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5266378-c33e-4afd-8457-422ac0b7b026 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:05.411878 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5266378-c33e-4afd-8457-422ac0b7b026 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:05.456844 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:05.487301 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 443/871] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:02:05 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 12:02:05.490757 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5266378-c33e-4afd-8457-422ac0b7b026 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:05.492363 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5266378-c33e-4afd-8457-422ac0b7b026 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:05.601079 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 429/872] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:02:05 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 12:02:05.608895 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70fa44-4658-4695-9a93-3f7b625266ee None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:05.629670 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5266378-c33e-4afd-8457-422ac0b7b026 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:05.689990 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:05.713402 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:05.714473 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70fa44-4658-4695-9a93-3f7b625266ee None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:05.750648 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5266378-c33e-4afd-8457-422ac0b7b026 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:05.762011 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cf71ee-e3f9-4d11-8274-7edb223c6327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:05.763483 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cf71ee-e3f9-4d11-8274-7edb223c6327 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:05.763929 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cf71ee-e3f9-4d11-8274-7edb223c6327 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:05.764773 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cf71ee-e3f9-4d11-8274-7edb223c6327 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:05.777627 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cf71ee-e3f9-4d11-8274-7edb223c6327 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:05.780008 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cf71ee-e3f9-4d11-8274-7edb223c6327 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:05.791020 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f006e09-2ba7-4ac5-a6ed-ab58c9ee61b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:05.791656 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f006e09-2ba7-4ac5-a6ed-ab58c9ee61b2 None admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:05.791997 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f006e09-2ba7-4ac5-a6ed-ab58c9ee61b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:05.792274 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f006e09-2ba7-4ac5-a6ed-ab58c9ee61b2 None admin] PATH_INFO: `/v3/services` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:05.792946 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f006e09-2ba7-4ac5-a6ed-ab58c9ee61b2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:05.794904 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f006e09-2ba7-4ac5-a6ed-ab58c9ee61b2 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:05.796269 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-93cf71ee-e3f9-4d11-8274-7edb223c6327 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:02:05.796269 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:05.796269 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:05.796269 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:05.796269 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:05.796269 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:05.796269 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:05.796269 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:05.796269 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:05.796269 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:05.796269 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:05.796269 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:05.796269 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:02:05.796269 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:02:05.796269 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:02:05.796269 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:02:05.797166 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:05.797166 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:05.797166 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:05.797166 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:05.797166 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:05.797166 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:05.797166 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:05.797166 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:05.797166 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:05.797166 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:05.797166 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:05.797166 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:05.797166 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:05.797166 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:05.797166 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:05.797166 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:05.797166 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:02:05.798068 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:02:05.798068 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:02:05.798068 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:05.799352 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 444/873] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:02:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:02:05.800161 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cf71ee-e3f9-4d11-8274-7edb223c6327 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:05.816369 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 430/874] 10.222.0.61 () {60 vars in 1141 bytes} [Fri Aug 26 12:02:05 2022] POST /identity/v3/services => generated 269 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 12:02:05.817535 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f006e09-2ba7-4ac5-a6ed-ab58c9ee61b2 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:05.942929 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cf71ee-e3f9-4d11-8274-7edb223c6327 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:05.966308 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f006e09-2ba7-4ac5-a6ed-ab58c9ee61b2 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:05.986695 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:05.993942 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b15b318-f887-4878-b880-bb5f2eec5f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:05.994577 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b15b318-f887-4878-b880-bb5f2eec5f20 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:05.994860 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b15b318-f887-4878-b880-bb5f2eec5f20 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:05.995133 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b15b318-f887-4878-b880-bb5f2eec5f20 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:05.995761 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b15b318-f887-4878-b880-bb5f2eec5f20 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:05.998134 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b15b318-f887-4878-b880-bb5f2eec5f20 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:06.006417 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-0b15b318-f887-4878-b880-bb5f2eec5f20 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 12:02:06.006417 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:06.006417 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 12:02:06.006417 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 12:02:06.006417 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 12:02:06.006417 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 12:02:06.006417 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 12:02:06.006417 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 12:02:06.006417 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 12:02:06.006417 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:06.006417 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 12:02:06.006417 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:06.006417 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:06.006417 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:06.006417 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:06.006417 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:06.006417 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:06.007461 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:06.007461 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:06.007461 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:06.007461 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:06.007461 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:06.007461 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:06.007461 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 12:02:06.007461 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 12:02:06.007461 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 12:02:06.007461 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 12:02:06.007461 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:06.007461 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:06.007461 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:06.007461 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:06.007461 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:06.007461 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:06.008593 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:06.008593 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:06.008593 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:06.008593 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:06.008593 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:06.008593 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:06.008593 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:06.008593 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:06.008593 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:06.008593 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:06.008593 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 12:02:06.008593 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 12:02:06.008593 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 12:02:06.008593 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:06.009406 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 445/875] 10.222.0.61 () {56 vars in 1126 bytes} [Fri Aug 26 12:02:05 2022] GET /identity/v3/services/object-store => generated 93 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:02:06.012696 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:06.015529 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b15b318-f887-4878-b880-bb5f2eec5f20 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:06.018433 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035fe8e2-0d2a-49f5-8c49-055553b43355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:06.019338 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035fe8e2-0d2a-49f5-8c49-055553b43355 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:06.019952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035fe8e2-0d2a-49f5-8c49-055553b43355 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:06.020457 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035fe8e2-0d2a-49f5-8c49-055553b43355 None admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:06.021423 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035fe8e2-0d2a-49f5-8c49-055553b43355 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:06.021731 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035fe8e2-0d2a-49f5-8c49-055553b43355 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:06.023099 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035fe8e2-0d2a-49f5-8c49-055553b43355 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:06.037436 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 431/876] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:02:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:06.042669 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035fe8e2-0d2a-49f5-8c49-055553b43355 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:06.144786 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b15b318-f887-4878-b880-bb5f2eec5f20 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:06.164865 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035fe8e2-0d2a-49f5-8c49-055553b43355 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:06.191915 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5455895c-e707-45fe-9ae4-d1b01c29a9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:06.192650 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5455895c-e707-45fe-9ae4-d1b01c29a9e3 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:06.193221 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5455895c-e707-45fe-9ae4-d1b01c29a9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:06.193501 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5455895c-e707-45fe-9ae4-d1b01c29a9e3 None admin] PATH_INFO: `/v3/services` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:06.194577 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5455895c-e707-45fe-9ae4-d1b01c29a9e3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:06.195149 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5455895c-e707-45fe-9ae4-d1b01c29a9e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:06.196800 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5455895c-e707-45fe-9ae4-d1b01c29a9e3 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:06.202613 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e296fd26-47f7-49cc-b09c-ba98cf6e6fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:06.203389 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e296fd26-47f7-49cc-b09c-ba98cf6e6fb9 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:06.203724 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e296fd26-47f7-49cc-b09c-ba98cf6e6fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:06.204114 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e296fd26-47f7-49cc-b09c-ba98cf6e6fb9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:06.207284 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 446/877] 10.222.0.61 () {56 vars in 1122 bytes} [Fri Aug 26 12:02:06 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:02:06.209164 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5455895c-e707-45fe-9ae4-d1b01c29a9e3 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:06.215171 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e296fd26-47f7-49cc-b09c-ba98cf6e6fb9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:06.217317 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e296fd26-47f7-49cc-b09c-ba98cf6e6fb9 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:06.229370 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-e296fd26-47f7-49cc-b09c-ba98cf6e6fb9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:02:06.229370 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:06.229370 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:06.229370 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:06.229370 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:06.229370 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:06.229370 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:06.229370 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:06.229370 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:06.229370 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:06.229370 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:06.229370 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:06.229370 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 12:02:06.229370 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 12:02:06.229370 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 12:02:06.230327 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 12:02:06.230327 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:06.230327 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:06.230327 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:06.230327 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:06.230327 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:06.230327 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:06.230327 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:06.230327 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:06.230327 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:06.230327 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:06.230327 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:06.230327 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:06.230327 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:06.230327 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:06.230327 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:06.230327 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:06.231293 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 12:02:06.231293 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 12:02:06.231293 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 12:02:06.231293 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 12:02:06.231293 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 12:02:06.231293 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 12:02:06.231293 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:02:06.231293 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:02:06.235261 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 432/878] 10.222.0.61 () {56 vars in 1111 bytes} [Fri Aug 26 12:02:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:02:06.236270 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e296fd26-47f7-49cc-b09c-ba98cf6e6fb9 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:06.289752 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:06.292008 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:06.346860 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e296fd26-47f7-49cc-b09c-ba98cf6e6fb9 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:06.348420 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5455895c-e707-45fe-9ae4-d1b01c29a9e3 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:06.383921 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e7ac14-33a6-4dba-a651-dfc9485409ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:06.384682 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e7ac14-33a6-4dba-a651-dfc9485409ed None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:06.385054 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e7ac14-33a6-4dba-a651-dfc9485409ed None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:06.385475 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e7ac14-33a6-4dba-a651-dfc9485409ed None admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:06.386203 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e7ac14-33a6-4dba-a651-dfc9485409ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:06.386667 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e7ac14-33a6-4dba-a651-dfc9485409ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:06.388805 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e7ac14-33a6-4dba-a651-dfc9485409ed None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:06.392828 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3590ebd-5292-4401-a697-7608a1957c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:06.393632 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3590ebd-5292-4401-a697-7608a1957c96 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:06.394054 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3590ebd-5292-4401-a697-7608a1957c96 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:06.394466 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3590ebd-5292-4401-a697-7608a1957c96 None admin] PATH_INFO: `/v3/services` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:06.395255 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3590ebd-5292-4401-a697-7608a1957c96 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:06.395699 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3590ebd-5292-4401-a697-7608a1957c96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:06.397690 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3590ebd-5292-4401-a697-7608a1957c96 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:06.411470 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 433/880] 10.222.0.61 () {56 vars in 1148 bytes} [Fri Aug 26 12:02:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:02:06.411665 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 447/880] 10.222.0.61 () {56 vars in 1122 bytes} [Fri Aug 26 12:02:06 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:06.412671 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e7ac14-33a6-4dba-a651-dfc9485409ed None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:06.530184 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e7ac14-33a6-4dba-a651-dfc9485409ed None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:06.544310 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:06.573074 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8800b1-6eee-472c-8b90-2d7f13b44498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:06.573685 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8800b1-6eee-472c-8b90-2d7f13b44498 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:06.573968 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8800b1-6eee-472c-8b90-2d7f13b44498 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:06.574244 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8800b1-6eee-472c-8b90-2d7f13b44498 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:06.574981 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8800b1-6eee-472c-8b90-2d7f13b44498 None admin] RBAC: Adding query filter params (role.id=519347d2bd2c4dc1aadb0b9f8c3c8e3d, scope.project.id=ecd68423a83a427681d44dbf266fce6e, user.id=755b04cae45746569bfbf92f2f213a58) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:06.575507 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8800b1-6eee-472c-8b90-2d7f13b44498 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:06.577172 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8800b1-6eee-472c-8b90-2d7f13b44498 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:06.583737 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 434/881] 10.222.0.61 () {56 vars in 1374 bytes} [Fri Aug 26 12:02:06 2022] GET /identity/v3/role_assignments?user.id=755b04cae45746569bfbf92f2f213a58&scope.project.id=ecd68423a83a427681d44dbf266fce6e&role.id=519347d2bd2c4dc1aadb0b9f8c3c8e3d => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:07.194131 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9778d29-51b6-42e6-bbe6-b96f3c2e8e7c None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:07.194644 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9778d29-51b6-42e6-bbe6-b96f3c2e8e7c None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:07.194749 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9778d29-51b6-42e6-bbe6-b96f3c2e8e7c None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:07.196297 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 448/882] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:02:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:02:07.201646 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f19d3c3-61b9-4805-a201-16bedb0df2d0 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:07.201934 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f19d3c3-61b9-4805-a201-16bedb0df2d0 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:07.202214 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f19d3c3-61b9-4805-a201-16bedb0df2d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:07.220833 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-0f19d3c3-61b9-4805-a201-16bedb0df2d0 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:02:07.222017 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-0f19d3c3-61b9-4805-a201-16bedb0df2d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:02:07.296790 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 435/883] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:02:07 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 12:02:07.569753 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9778d29-51b6-42e6-bbe6-b96f3c2e8e7c None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:07.677361 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9778d29-51b6-42e6-bbe6-b96f3c2e8e7c None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:07.701439 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:07.713456 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163ece26-4e9f-4002-a317-6bd240a01656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:07.714375 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ece26-4e9f-4002-a317-6bd240a01656 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:07.714985 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ece26-4e9f-4002-a317-6bd240a01656 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:07.716304 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ece26-4e9f-4002-a317-6bd240a01656 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:07.717055 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163ece26-4e9f-4002-a317-6bd240a01656 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:07.718428 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163ece26-4e9f-4002-a317-6bd240a01656 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:07.727241 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-163ece26-4e9f-4002-a317-6bd240a01656 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 12:02:07.727241 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:07.727241 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 12:02:07.727241 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 12:02:07.727241 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 12:02:07.727241 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 12:02:07.727241 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 12:02:07.727241 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 12:02:07.727241 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 12:02:07.727241 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:07.727241 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 12:02:07.727241 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:07.727241 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:07.727241 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:07.727241 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:07.727241 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:07.727241 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:07.728239 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:07.728239 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:07.728239 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:07.728239 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:07.728239 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:07.728239 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:07.728239 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 12:02:07.728239 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 12:02:07.728239 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 12:02:07.728239 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 12:02:07.728239 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:07.728239 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:07.728239 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:07.728239 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:07.728239 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:07.728239 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:07.729387 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:07.729387 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:07.729387 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:07.729387 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:07.729387 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:07.729387 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:07.729387 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:07.729387 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:07.729387 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:07.729387 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:07.729387 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 12:02:07.729387 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 12:02:07.729387 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 12:02:07.729387 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:07.729387 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 449/884] 10.222.0.61 () {56 vars in 1114 bytes} [Fri Aug 26 12:02:07 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:02:07.734785 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f19d3c3-61b9-4805-a201-16bedb0df2d0 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:07.840890 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f19d3c3-61b9-4805-a201-16bedb0df2d0 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:07.844498 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:07.848801 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f0a3eb-24f0-49c9-85af-a5b5558e12eb None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:07.849193 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f0a3eb-24f0-49c9-85af-a5b5558e12eb None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:07.851058 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f0a3eb-24f0-49c9-85af-a5b5558e12eb None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:07.851169 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 450/885] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:02:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:02:07.856634 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d7e265-18e3-434c-bad3-d5c181f0a9aa None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:07.856985 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d7e265-18e3-434c-bad3-d5c181f0a9aa None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:07.857325 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d7e265-18e3-434c-bad3-d5c181f0a9aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:07.876055 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-c4d7e265-18e3-434c-bad3-d5c181f0a9aa None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:02:07.877324 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-c4d7e265-18e3-434c-bad3-d5c181f0a9aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:02:07.882848 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b39736-6313-46bb-9d3b-c8d2a7ff8db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:07.883879 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b39736-6313-46bb-9d3b-c8d2a7ff8db9 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:07.884160 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b39736-6313-46bb-9d3b-c8d2a7ff8db9 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:07.884433 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b39736-6313-46bb-9d3b-c8d2a7ff8db9 None admin] PATH_INFO: `/v3/services` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:07.885060 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b39736-6313-46bb-9d3b-c8d2a7ff8db9 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:07.885359 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b39736-6313-46bb-9d3b-c8d2a7ff8db9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:07.887166 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b39736-6313-46bb-9d3b-c8d2a7ff8db9 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:07.893696 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 436/886] 10.222.0.61 () {56 vars in 1114 bytes} [Fri Aug 26 12:02:07 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:07.899508 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b39736-6313-46bb-9d3b-c8d2a7ff8db9 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:07.955574 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 451/887] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:02:07 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 12:02:08.011579 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b39736-6313-46bb-9d3b-c8d2a7ff8db9 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:08.046597 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a435b1f-0a99-4de5-9fc1-6085726c1608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:08.047224 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a435b1f-0a99-4de5-9fc1-6085726c1608 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:08.047512 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a435b1f-0a99-4de5-9fc1-6085726c1608 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:08.047786 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a435b1f-0a99-4de5-9fc1-6085726c1608 None admin] PATH_INFO: `/v3/services` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:08.048413 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a435b1f-0a99-4de5-9fc1-6085726c1608 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:08.048717 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a435b1f-0a99-4de5-9fc1-6085726c1608 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:08.050082 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a435b1f-0a99-4de5-9fc1-6085726c1608 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:08.055359 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 437/888] 10.222.0.61 () {56 vars in 1114 bytes} [Fri Aug 26 12:02:07 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:08.061649 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d7e265-18e3-434c-bad3-d5c181f0a9aa None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:08.167658 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d7e265-18e3-434c-bad3-d5c181f0a9aa None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:08.185091 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:08.203714 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53561d30-8249-44a1-9fd4-ca94257ed3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:08.204441 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53561d30-8249-44a1-9fd4-ca94257ed3b9 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:08.204596 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53561d30-8249-44a1-9fd4-ca94257ed3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:08.204841 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53561d30-8249-44a1-9fd4-ca94257ed3b9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:08.205547 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53561d30-8249-44a1-9fd4-ca94257ed3b9 None admin] RBAC: Adding query filter params (interface=public, service_id=86421df984a540328c913714ac42d40e, region_id=RegionOne) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:08.205856 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53561d30-8249-44a1-9fd4-ca94257ed3b9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:08.207231 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53561d30-8249-44a1-9fd4-ca94257ed3b9 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:08.215732 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 452/889] 10.222.0.61 () {56 vars in 1251 bytes} [Fri Aug 26 12:02:08 2022] GET /identity/v3/endpoints?service_id=86421df984a540328c913714ac42d40e&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:08.220844 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a435b1f-0a99-4de5-9fc1-6085726c1608 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:08.229270 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53561d30-8249-44a1-9fd4-ca94257ed3b9 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:08.254381 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:08.331806 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a435b1f-0a99-4de5-9fc1-6085726c1608 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:08.354471 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53561d30-8249-44a1-9fd4-ca94257ed3b9 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:08.370338 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49300f88-21b7-43cd-a99d-ce94e6ebf26a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:08.371476 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49300f88-21b7-43cd-a99d-ce94e6ebf26a None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:08.371985 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49300f88-21b7-43cd-a99d-ce94e6ebf26a None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:08.372864 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49300f88-21b7-43cd-a99d-ce94e6ebf26a None admin] PATH_INFO: `/v3/services` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:08.373776 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49300f88-21b7-43cd-a99d-ce94e6ebf26a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:08.375709 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49300f88-21b7-43cd-a99d-ce94e6ebf26a None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:08.391265 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 438/890] 10.222.0.61 () {56 vars in 1087 bytes} [Fri Aug 26 12:02:08 2022] GET /identity/v3/services => generated 1268 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:02:08.403835 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcee73b-d88c-47aa-b4cc-d364628c6bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:08.404682 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcee73b-d88c-47aa-b4cc-d364628c6bda None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:08.404959 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcee73b-d88c-47aa-b4cc-d364628c6bda None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:08.405227 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcee73b-d88c-47aa-b4cc-d364628c6bda None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:08.405824 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcee73b-d88c-47aa-b4cc-d364628c6bda None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:08.407256 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcee73b-d88c-47aa-b4cc-d364628c6bda None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:08.416969 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-2dcee73b-d88c-47aa-b4cc-d364628c6bda None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 12:02:08.416969 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:08.416969 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 12:02:08.416969 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 12:02:08.416969 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 12:02:08.416969 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 12:02:08.416969 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 12:02:08.416969 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 12:02:08.416969 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 12:02:08.416969 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:08.416969 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 12:02:08.416969 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:08.416969 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:08.416969 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:08.416969 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:08.416969 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:08.418056 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:08.418056 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:08.418056 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:08.418056 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:08.418056 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:08.418056 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:08.418056 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:08.418056 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 12:02:08.418056 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 12:02:08.418056 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 12:02:08.418056 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 12:02:08.418056 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:08.418056 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:08.418056 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:08.418056 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:08.418056 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:08.418056 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:08.419125 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:08.419125 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:08.419125 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:08.419125 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:08.419125 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:08.419125 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:08.419125 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:08.419125 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:08.419125 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:08.419125 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:08.419125 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 12:02:08.419125 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 12:02:08.419125 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 12:02:08.419125 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:08.419125 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 453/891] 10.222.0.61 () {56 vars in 1132 bytes} [Fri Aug 26 12:02:08 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 12:02:08.424364 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49300f88-21b7-43cd-a99d-ce94e6ebf26a None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:08.495572 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fcdd90-ee3c-43f9-a11e-7676096a10b5 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:08.495902 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fcdd90-ee3c-43f9-a11e-7676096a10b5 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:08.496190 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fcdd90-ee3c-43f9-a11e-7676096a10b5 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:08.498018 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 454/892] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:02:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:02:08.503726 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb96244-dbe8-434c-9abd-1c694cabde18 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:08.504018 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb96244-dbe8-434c-9abd-1c694cabde18 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:08.504272 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb96244-dbe8-434c-9abd-1c694cabde18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:08.527066 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-bdb96244-dbe8-434c-9abd-1c694cabde18 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:02:08.529239 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-bdb96244-dbe8-434c-9abd-1c694cabde18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:02:08.529797 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:08.536793 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49300f88-21b7-43cd-a99d-ce94e6ebf26a None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:08.580740 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbc2acf-c9f3-4f2d-8b6c-2fd5fb2f88f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:08.581362 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbc2acf-c9f3-4f2d-8b6c-2fd5fb2f88f5 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:08.581646 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbc2acf-c9f3-4f2d-8b6c-2fd5fb2f88f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:08.581914 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbc2acf-c9f3-4f2d-8b6c-2fd5fb2f88f5 None admin] PATH_INFO: `/v3/services` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:08.582548 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbc2acf-c9f3-4f2d-8b6c-2fd5fb2f88f5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:08.583088 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:08.583540 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbc2acf-c9f3-4f2d-8b6c-2fd5fb2f88f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:08.584935 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbc2acf-c9f3-4f2d-8b6c-2fd5fb2f88f5 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:08.592776 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 439/893] 10.222.0.61 () {56 vars in 1126 bytes} [Fri Aug 26 12:02:08 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:08.599256 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbc2acf-c9f3-4f2d-8b6c-2fd5fb2f88f5 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:08.624144 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 455/894] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:02:08 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 12:02:08.664869 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bbd041-caa5-408b-81e6-569372ebd19e None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:08.665178 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bbd041-caa5-408b-81e6-569372ebd19e None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:08.665420 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bbd041-caa5-408b-81e6-569372ebd19e None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:08.671356 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 456/895] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:02:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 12:02:08.676558 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad7310f-7c89-408f-9509-32206b735d24 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:08.676839 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad7310f-7c89-408f-9509-32206b735d24 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:08.677172 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad7310f-7c89-408f-9509-32206b735d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:08.696809 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-4ad7310f-7c89-408f-9509-32206b735d24 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:02:08.697997 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-4ad7310f-7c89-408f-9509-32206b735d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:02:08.714187 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbc2acf-c9f3-4f2d-8b6c-2fd5fb2f88f5 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:08.755788 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f26ab64-a3e6-451c-b5ad-022350a4e0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:08.756445 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f26ab64-a3e6-451c-b5ad-022350a4e0e3 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:08.756718 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f26ab64-a3e6-451c-b5ad-022350a4e0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:08.756995 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f26ab64-a3e6-451c-b5ad-022350a4e0e3 None admin] PATH_INFO: `/v3/services` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:08.757625 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f26ab64-a3e6-451c-b5ad-022350a4e0e3 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:08.757927 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f26ab64-a3e6-451c-b5ad-022350a4e0e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:08.759677 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f26ab64-a3e6-451c-b5ad-022350a4e0e3 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:08.765277 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 440/896] 10.222.0.61 () {56 vars in 1126 bytes} [Fri Aug 26 12:02:08 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:02:08.771399 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f26ab64-a3e6-451c-b5ad-022350a4e0e3 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:08.783133 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 457/897] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:02:08 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 12:02:08.803230 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:08.880463 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f26ab64-a3e6-451c-b5ad-022350a4e0e3 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:08.910555 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad7310f-7c89-408f-9509-32206b735d24 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:08.916898 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe21c0e-a13d-423a-a5f7-6d882fe4f7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:08.917571 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe21c0e-a13d-423a-a5f7-6d882fe4f7cd None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:08.917855 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe21c0e-a13d-423a-a5f7-6d882fe4f7cd None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:08.918131 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe21c0e-a13d-423a-a5f7-6d882fe4f7cd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:08.919467 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe21c0e-a13d-423a-a5f7-6d882fe4f7cd None admin] RBAC: Adding query filter params (interface=public, service_id=58069726e0cb430a87fb56b361a9acb5, region_id=RegionOne) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:08.919782 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe21c0e-a13d-423a-a5f7-6d882fe4f7cd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:08.921274 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe21c0e-a13d-423a-a5f7-6d882fe4f7cd None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:08.927475 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 441/898] 10.222.0.61 () {56 vars in 1251 bytes} [Fri Aug 26 12:02:08 2022] GET /identity/v3/endpoints?service_id=58069726e0cb430a87fb56b361a9acb5&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:08.932782 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe21c0e-a13d-423a-a5f7-6d882fe4f7cd None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:09.040815 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:09.042107 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad7310f-7c89-408f-9509-32206b735d24 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:09.045627 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:09.061469 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe21c0e-a13d-423a-a5f7-6d882fe4f7cd None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:09.082227 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eb93f6-033f-484e-b9ed-0de6e5029ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:09.082836 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eb93f6-033f-484e-b9ed-0de6e5029ee1 None admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:09.083121 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eb93f6-033f-484e-b9ed-0de6e5029ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:09.083491 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eb93f6-033f-484e-b9ed-0de6e5029ee1 None admin] PATH_INFO: `/v3/services` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:09.084170 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eb93f6-033f-484e-b9ed-0de6e5029ee1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:09.085540 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eb93f6-033f-484e-b9ed-0de6e5029ee1 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:09.100001 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 458/899] 10.222.0.61 () {60 vars in 1141 bytes} [Fri Aug 26 12:02:08 2022] POST /identity/v3/services => generated 241 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 12:02:09.100816 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eb93f6-033f-484e-b9ed-0de6e5029ee1 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:09.114439 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d79620e-8eff-42cc-935f-2ab870f3b540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:09.115903 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d79620e-8eff-42cc-935f-2ab870f3b540 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:09.116644 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d79620e-8eff-42cc-935f-2ab870f3b540 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:09.117183 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d79620e-8eff-42cc-935f-2ab870f3b540 None admin] PATH_INFO: `/v3/services` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:09.118005 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d79620e-8eff-42cc-935f-2ab870f3b540 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:09.120728 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d79620e-8eff-42cc-935f-2ab870f3b540 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:09.139498 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 442/900] 10.222.0.61 () {56 vars in 1087 bytes} [Fri Aug 26 12:02:08 2022] GET /identity/v3/services => generated 1497 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 12:02:09.240479 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eb93f6-033f-484e-b9ed-0de6e5029ee1 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:09.287171 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc60b15e-37d2-487a-b9be-7f554c985170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:09.287849 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc60b15e-37d2-487a-b9be-7f554c985170 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:09.288127 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc60b15e-37d2-487a-b9be-7f554c985170 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:09.288400 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc60b15e-37d2-487a-b9be-7f554c985170 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:09.301759 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc60b15e-37d2-487a-b9be-7f554c985170 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:02:09.306996 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-fc60b15e-37d2-487a-b9be-7f554c985170 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 12:02:09.306996 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:09.306996 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:09.306996 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:09.306996 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:09.306996 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:09.306996 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:09.306996 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:09.306996 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:09.306996 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:09.306996 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:09.306996 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:09.306996 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:02:09.306996 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:02:09.306996 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:02:09.306996 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:02:09.307983 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:02:09.307983 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:02:09.307983 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:02:09.307983 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:02:09.307983 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:09.307983 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:09.307983 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:09.307983 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:09.307983 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:09.307983 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:09.307983 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:09.307983 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:09.307983 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:09.307983 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:09.307983 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:09.307983 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:09.308833 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:09.308833 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:09.308833 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:09.308833 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:09.308833 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:02:09.308833 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:02:09.308833 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:02:09.308833 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:02:09.308833 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:02:09.308833 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:02:09.308833 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 12:02:09.308833 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:09.308833 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 459/901] 10.222.0.61 () {56 vars in 1120 bytes} [Fri Aug 26 12:02:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:02:09.314501 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d79620e-8eff-42cc-935f-2ab870f3b540 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:09.423232 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d79620e-8eff-42cc-935f-2ab870f3b540 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:09.458731 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88712e3b-2b9c-460c-b40b-a968c5c2929d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:09.459383 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88712e3b-2b9c-460c-b40b-a968c5c2929d None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:09.459664 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88712e3b-2b9c-460c-b40b-a968c5c2929d None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:09.459939 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88712e3b-2b9c-460c-b40b-a968c5c2929d None admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:09.460606 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88712e3b-2b9c-460c-b40b-a968c5c2929d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:09.460910 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88712e3b-2b9c-460c-b40b-a968c5c2929d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:09.462348 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88712e3b-2b9c-460c-b40b-a968c5c2929d None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:09.472233 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 443/902] 10.222.0.61 () {56 vars in 1115 bytes} [Fri Aug 26 12:02:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:09.477777 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc60b15e-37d2-487a-b9be-7f554c985170 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:09.506737 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:09.595354 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc60b15e-37d2-487a-b9be-7f554c985170 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:09.630344 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d5868e-e177-4f4d-932d-6c6257432058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:09.630974 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d5868e-e177-4f4d-932d-6c6257432058 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:09.631283 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d5868e-e177-4f4d-932d-6c6257432058 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:09.631553 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d5868e-e177-4f4d-932d-6c6257432058 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:09.643857 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d5868e-e177-4f4d-932d-6c6257432058 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:09.645380 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d5868e-e177-4f4d-932d-6c6257432058 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:09.660087 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-71d5868e-e177-4f4d-932d-6c6257432058 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:02:09.660087 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:09.660087 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:09.660087 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:09.660087 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:09.660087 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:09.660087 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:09.660087 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:09.660087 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:09.660087 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:09.660087 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:09.660087 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:09.660087 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:02:09.660087 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:02:09.660087 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:02:09.660087 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:02:09.661084 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:09.661084 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:09.661084 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:09.661084 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:09.661084 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:09.661084 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:09.661084 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:09.661084 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:09.661084 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:09.661084 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:09.661084 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:09.661084 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:09.661084 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:09.661084 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:09.661084 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:09.661084 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:09.661084 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:02:09.661994 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:02:09.661994 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:02:09.661994 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:09.661994 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 460/903] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:02:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:02:09.668473 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88712e3b-2b9c-460c-b40b-a968c5c2929d None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:09.683495 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:09.774309 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88712e3b-2b9c-460c-b40b-a968c5c2929d None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:09.808856 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91540cc1-80d6-4c5d-a71d-a52fd7ed4e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:09.809610 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91540cc1-80d6-4c5d-a71d-a52fd7ed4e66 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:09.809893 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91540cc1-80d6-4c5d-a71d-a52fd7ed4e66 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:09.810165 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91540cc1-80d6-4c5d-a71d-a52fd7ed4e66 None admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:09.810810 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91540cc1-80d6-4c5d-a71d-a52fd7ed4e66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:09.811110 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91540cc1-80d6-4c5d-a71d-a52fd7ed4e66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:09.812509 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91540cc1-80d6-4c5d-a71d-a52fd7ed4e66 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:09.825001 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 444/904] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:02:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:02:09.831311 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d5868e-e177-4f4d-932d-6c6257432058 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:09.918294 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:09.943050 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d5868e-e177-4f4d-932d-6c6257432058 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:09.976906 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad64c30-466b-46a8-9dae-8fd388da049f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:09.977582 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad64c30-466b-46a8-9dae-8fd388da049f None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:09.977862 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad64c30-466b-46a8-9dae-8fd388da049f None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:09.978128 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad64c30-466b-46a8-9dae-8fd388da049f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:09.986737 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad64c30-466b-46a8-9dae-8fd388da049f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:09.988239 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad64c30-466b-46a8-9dae-8fd388da049f None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:10.000377 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-3ad64c30-466b-46a8-9dae-8fd388da049f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 12:02:10.000377 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:10.000377 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:10.000377 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:10.000377 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:10.000377 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:10.000377 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:10.000377 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:10.000377 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:10.000377 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:10.000377 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:10.000377 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:10.000377 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 12:02:10.000377 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 12:02:10.000377 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 12:02:10.000377 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 12:02:10.001336 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:10.001336 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:10.001336 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 12:02:10.001336 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:02:10.001336 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 12:02:10.001336 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:02:10.001336 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:10.001336 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:10.001336 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:10.001336 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:10.001336 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:10.001336 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:10.001336 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:10.001336 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:10.001336 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:10.001336 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:10.001336 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:10.001336 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:10.002302 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:10.002302 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:10.002302 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 12:02:10.002302 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 12:02:10.002302 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 12:02:10.002302 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 12:02:10.002302 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 12:02:10.002302 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 12:02:10.002302 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 12:02:10.002302 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:10.002302 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 461/905] 10.222.0.61 () {56 vars in 1117 bytes} [Fri Aug 26 12:02:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:02:10.009419 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91540cc1-80d6-4c5d-a71d-a52fd7ed4e66 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:10.082605 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:10.113810 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91540cc1-80d6-4c5d-a71d-a52fd7ed4e66 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:10.149853 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc30e40d-31bc-417f-9aca-a913fa2fdea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:10.150457 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc30e40d-31bc-417f-9aca-a913fa2fdea7 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:10.150736 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc30e40d-31bc-417f-9aca-a913fa2fdea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:10.151007 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc30e40d-31bc-417f-9aca-a913fa2fdea7 None admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:10.151760 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc30e40d-31bc-417f-9aca-a913fa2fdea7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:10.152063 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc30e40d-31bc-417f-9aca-a913fa2fdea7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:10.153602 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc30e40d-31bc-417f-9aca-a913fa2fdea7 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:10.171951 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 445/906] 10.222.0.61 () {56 vars in 1149 bytes} [Fri Aug 26 12:02:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:10.177548 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad64c30-466b-46a8-9dae-8fd388da049f None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:10.286351 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad64c30-466b-46a8-9dae-8fd388da049f None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:10.321800 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b845eb2-d09e-4f22-bf44-5bed2380a9bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:10.322400 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b845eb2-d09e-4f22-bf44-5bed2380a9bc None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:10.322676 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b845eb2-d09e-4f22-bf44-5bed2380a9bc None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:10.322947 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b845eb2-d09e-4f22-bf44-5bed2380a9bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:10.326614 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:10.397946 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b845eb2-d09e-4f22-bf44-5bed2380a9bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:10.399506 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b845eb2-d09e-4f22-bf44-5bed2380a9bc None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:10.411447 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd030e8f-3822-4d2c-9a00-a11ff21b9e4a None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:10.411738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd030e8f-3822-4d2c-9a00-a11ff21b9e4a None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:10.412013 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd030e8f-3822-4d2c-9a00-a11ff21b9e4a None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:10.413607 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 446/907] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:02:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:02:10.418358 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae6551d-f282-4a4c-9dd3-134ea66688cf None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:10.419390 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae6551d-f282-4a4c-9dd3-134ea66688cf None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:10.419749 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae6551d-f282-4a4c-9dd3-134ea66688cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:10.437242 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-8ae6551d-f282-4a4c-9dd3-134ea66688cf None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:02:10.438429 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-8ae6551d-f282-4a4c-9dd3-134ea66688cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:02:10.502045 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 462/908] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:02:10 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 12:02:10.507623 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b845eb2-d09e-4f22-bf44-5bed2380a9bc None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:10.519313 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 447/909] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:02:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 12:02:10.544661 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:10.611169 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b845eb2-d09e-4f22-bf44-5bed2380a9bc None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:10.646656 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67b3186-d19a-4139-ae16-352bf991215c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:10.647627 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67b3186-d19a-4139-ae16-352bf991215c None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:10.648157 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67b3186-d19a-4139-ae16-352bf991215c None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:10.648582 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67b3186-d19a-4139-ae16-352bf991215c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:10.659804 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67b3186-d19a-4139-ae16-352bf991215c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:10.661459 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67b3186-d19a-4139-ae16-352bf991215c None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:10.676511 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-c67b3186-d19a-4139-ae16-352bf991215c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:02:10.676511 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:10.676511 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:10.676511 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:10.676511 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:10.676511 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:10.676511 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:10.676511 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:10.676511 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:10.676511 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:10.676511 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:10.676511 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:10.676511 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:02:10.676511 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:02:10.676511 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:02:10.676511 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:02:10.677733 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:10.677733 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:10.677733 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:10.677733 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:10.677733 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:10.677733 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:10.677733 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:10.677733 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:10.677733 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:10.677733 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:10.677733 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:10.677733 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:10.677733 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:10.677733 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:10.677733 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:10.677733 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:10.677733 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:02:10.678933 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:02:10.678933 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:02:10.678933 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:10.680633 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 463/910] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:02:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:02:10.686549 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae6551d-f282-4a4c-9dd3-134ea66688cf None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:10.749289 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:10.798232 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae6551d-f282-4a4c-9dd3-134ea66688cf None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:10.799256 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67b3186-d19a-4139-ae16-352bf991215c None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:10.842774 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f004de0f-0b55-4342-96ed-b5abbda865ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:10.844325 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f004de0f-0b55-4342-96ed-b5abbda865ed None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:10.844833 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f004de0f-0b55-4342-96ed-b5abbda865ed None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:10.845321 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f004de0f-0b55-4342-96ed-b5abbda865ed None admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:10.846656 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f004de0f-0b55-4342-96ed-b5abbda865ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:10.847227 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f004de0f-0b55-4342-96ed-b5abbda865ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:10.849476 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f004de0f-0b55-4342-96ed-b5abbda865ed None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:10.863073 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 448/911] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:02:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:10.868627 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f004de0f-0b55-4342-96ed-b5abbda865ed None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:10.900258 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:10.915457 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67b3186-d19a-4139-ae16-352bf991215c None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:10.957211 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6262e5-a6fb-44cd-9483-599dcec515ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:10.957943 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6262e5-a6fb-44cd-9483-599dcec515ea None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:10.958287 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6262e5-a6fb-44cd-9483-599dcec515ea None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:10.958631 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6262e5-a6fb-44cd-9483-599dcec515ea None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:10.959334 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6262e5-a6fb-44cd-9483-599dcec515ea None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:10.960768 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6262e5-a6fb-44cd-9483-599dcec515ea None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:10.971189 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-fc6262e5-a6fb-44cd-9483-599dcec515ea None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 12:02:10.971189 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:10.971189 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 12:02:10.971189 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 12:02:10.971189 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 12:02:10.971189 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 12:02:10.971189 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 12:02:10.971189 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 12:02:10.971189 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 12:02:10.971189 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:10.971189 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 12:02:10.971189 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:10.971189 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:10.971189 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:10.971189 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:10.971189 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:10.971189 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:10.972368 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:10.972368 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:10.972368 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:10.972368 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:10.972368 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:10.972368 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:10.972368 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 12:02:10.972368 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 12:02:10.972368 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 12:02:10.972368 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 12:02:10.972368 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:10.972368 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:10.972368 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:10.972368 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:10.972368 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:10.972368 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:10.973225 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:10.973225 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:10.973225 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:10.973225 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:10.973225 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:10.973225 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:10.973225 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:10.973225 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:10.973225 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:10.973225 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:10.973225 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 12:02:10.973225 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 12:02:10.973225 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 26 12:02:10.973225 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:10.973225 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 464/912] 10.222.0.61 () {56 vars in 1114 bytes} [Fri Aug 26 12:02:10 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 12:02:10.980459 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6262e5-a6fb-44cd-9483-599dcec515ea None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:10.995356 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f004de0f-0b55-4342-96ed-b5abbda865ed None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:11.016950 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:11.037376 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021a3b81-ee8d-4986-b07d-88e77746854a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:11.038010 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021a3b81-ee8d-4986-b07d-88e77746854a None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:11.038289 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021a3b81-ee8d-4986-b07d-88e77746854a None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:11.038568 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021a3b81-ee8d-4986-b07d-88e77746854a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:11.047418 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021a3b81-ee8d-4986-b07d-88e77746854a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:11.049185 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021a3b81-ee8d-4986-b07d-88e77746854a None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:11.063126 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-021a3b81-ee8d-4986-b07d-88e77746854a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:02:11.063126 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:11.063126 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:11.063126 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:11.063126 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:11.063126 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:11.063126 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:11.063126 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:11.063126 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:11.063126 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:11.063126 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:11.063126 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:11.063126 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 12:02:11.063126 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 12:02:11.063126 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 12:02:11.064119 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 12:02:11.064119 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:11.064119 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:11.064119 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:11.064119 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:11.064119 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:11.064119 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:11.064119 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:11.064119 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:11.064119 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:11.064119 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:11.064119 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:11.064119 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:11.064119 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:11.064119 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:11.064119 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:11.064119 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:11.065045 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 12:02:11.065045 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 12:02:11.065045 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 12:02:11.065045 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 12:02:11.065045 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 12:02:11.065045 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 12:02:11.065045 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:02:11.065045 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:02:11.067614 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 449/913] 10.222.0.61 () {56 vars in 1111 bytes} [Fri Aug 26 12:02:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:02:11.074307 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021a3b81-ee8d-4986-b07d-88e77746854a None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:11.109519 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6262e5-a6fb-44cd-9483-599dcec515ea None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:11.148216 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c733aeb-d697-4810-8fab-5aa49f46e6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:11.148724 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c733aeb-d697-4810-8fab-5aa49f46e6f6 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:11.149004 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c733aeb-d697-4810-8fab-5aa49f46e6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:11.149276 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c733aeb-d697-4810-8fab-5aa49f46e6f6 None admin] PATH_INFO: `/v3/services` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:11.149964 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c733aeb-d697-4810-8fab-5aa49f46e6f6 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:11.150320 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c733aeb-d697-4810-8fab-5aa49f46e6f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:11.151741 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c733aeb-d697-4810-8fab-5aa49f46e6f6 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:11.162178 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 465/914] 10.222.0.61 () {56 vars in 1114 bytes} [Fri Aug 26 12:02:10 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:02:11.163229 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c733aeb-d697-4810-8fab-5aa49f46e6f6 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:11.183588 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:11.217088 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021a3b81-ee8d-4986-b07d-88e77746854a None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:11.260695 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0cdd6-4531-4c1a-94f2-d87a11d46a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:11.261381 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0cdd6-4531-4c1a-94f2-d87a11d46a8b None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:11.261586 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0cdd6-4531-4c1a-94f2-d87a11d46a8b None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:11.261856 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0cdd6-4531-4c1a-94f2-d87a11d46a8b None admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:11.263352 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec0cdd6-4531-4c1a-94f2-d87a11d46a8b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:11.263736 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec0cdd6-4531-4c1a-94f2-d87a11d46a8b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:11.265180 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec0cdd6-4531-4c1a-94f2-d87a11d46a8b None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:11.272565 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c733aeb-d697-4810-8fab-5aa49f46e6f6 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:11.282850 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 450/915] 10.222.0.61 () {56 vars in 1148 bytes} [Fri Aug 26 12:02:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:11.282850 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb6afe-8bf6-4fe9-8a03-8a91a7419098 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:11.282850 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb6afe-8bf6-4fe9-8a03-8a91a7419098 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:11.282850 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eb6afe-8bf6-4fe9-8a03-8a91a7419098 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:11.282850 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 451/916] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:02:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:02:11.284322 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eb6afe-8bf6-4fe9-8a03-8a91a7419098 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:11.313012 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c889c633-7f16-4115-8403-5bc56f62ae65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:11.313611 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c889c633-7f16-4115-8403-5bc56f62ae65 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:11.313889 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c889c633-7f16-4115-8403-5bc56f62ae65 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:11.314159 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c889c633-7f16-4115-8403-5bc56f62ae65 None admin] PATH_INFO: `/v3/services` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:11.314776 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c889c633-7f16-4115-8403-5bc56f62ae65 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:11.315076 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c889c633-7f16-4115-8403-5bc56f62ae65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:11.316488 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c889c633-7f16-4115-8403-5bc56f62ae65 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:11.321309 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 466/917] 10.222.0.61 () {56 vars in 1114 bytes} [Fri Aug 26 12:02:11 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:02:11.322814 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca4b12-4c1e-4816-8ed5-f298034d6467 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:11.323444 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca4b12-4c1e-4816-8ed5-f298034d6467 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:11.323725 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ca4b12-4c1e-4816-8ed5-f298034d6467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:11.329038 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:11.341070 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-38ca4b12-4c1e-4816-8ed5-f298034d6467 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:02:11.342245 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-38ca4b12-4c1e-4816-8ed5-f298034d6467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:02:11.390698 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eb6afe-8bf6-4fe9-8a03-8a91a7419098 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:11.407976 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 467/918] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:02:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 12:02:11.409136 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33161d1-180b-4668-b47d-b0b7e7c7f5ba None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:11.409419 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33161d1-180b-4668-b47d-b0b7e7c7f5ba None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:11.409694 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33161d1-180b-4668-b47d-b0b7e7c7f5ba None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:11.411058 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 468/919] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:02:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:02:11.411602 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33161d1-180b-4668-b47d-b0b7e7c7f5ba None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:11.423963 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:11.425767 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c44ff9c-f7ce-41fb-8ba2-533380d594ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:11.426406 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c44ff9c-f7ce-41fb-8ba2-533380d594ca None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:11.426741 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c44ff9c-f7ce-41fb-8ba2-533380d594ca None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:11.427056 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c44ff9c-f7ce-41fb-8ba2-533380d594ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:11.427934 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c44ff9c-f7ce-41fb-8ba2-533380d594ca None admin] RBAC: Adding query filter params (role.id=ebae7fe0ca914cd1a0106219ed37c042, scope.project.id=ecd68423a83a427681d44dbf266fce6e, user.id=755b04cae45746569bfbf92f2f213a58) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:11.428378 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c44ff9c-f7ce-41fb-8ba2-533380d594ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:11.429805 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c44ff9c-f7ce-41fb-8ba2-533380d594ca None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:11.443258 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 452/920] 10.222.0.61 () {56 vars in 1374 bytes} [Fri Aug 26 12:02:11 2022] GET /identity/v3/role_assignments?user.id=755b04cae45746569bfbf92f2f213a58&scope.project.id=ecd68423a83a427681d44dbf266fce6e&role.id=ebae7fe0ca914cd1a0106219ed37c042 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:11.443258 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740d3f88-1610-4b9e-bf63-7eff5614454a None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:11.443258 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740d3f88-1610-4b9e-bf63-7eff5614454a None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:11.443258 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740d3f88-1610-4b9e-bf63-7eff5614454a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:11.463539 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-740d3f88-1610-4b9e-bf63-7eff5614454a None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:02:11.464755 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-740d3f88-1610-4b9e-bf63-7eff5614454a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:02:11.551196 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33161d1-180b-4668-b47d-b0b7e7c7f5ba None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:11.557189 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 453/921] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:02:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 12:02:11.596004 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806fff32-6a48-4f12-83f3-a19be9008bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:11.596623 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806fff32-6a48-4f12-83f3-a19be9008bbb None admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:11.596909 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806fff32-6a48-4f12-83f3-a19be9008bbb None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:11.597180 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806fff32-6a48-4f12-83f3-a19be9008bbb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:11.597856 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806fff32-6a48-4f12-83f3-a19be9008bbb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:11.599331 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806fff32-6a48-4f12-83f3-a19be9008bbb None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:11.621196 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 469/922] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:02:11 2022] POST /identity/v3/endpoints => generated 346 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:02:11.695790 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740d3f88-1610-4b9e-bf63-7eff5614454a None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:11.730601 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:11.806162 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740d3f88-1610-4b9e-bf63-7eff5614454a None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:11.848139 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806fff32-6a48-4f12-83f3-a19be9008bbb None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:11.857633 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8854118-53ad-4566-b5bd-10d83a6654b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:11.858328 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8854118-53ad-4566-b5bd-10d83a6654b9 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:11.858609 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8854118-53ad-4566-b5bd-10d83a6654b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:11.858877 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8854118-53ad-4566-b5bd-10d83a6654b9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:11.859492 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8854118-53ad-4566-b5bd-10d83a6654b9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:11.860867 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8854118-53ad-4566-b5bd-10d83a6654b9 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:11.869994 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-f8854118-53ad-4566-b5bd-10d83a6654b9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 12:02:11.869994 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:11.869994 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 12:02:11.869994 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 12:02:11.869994 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 12:02:11.869994 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 12:02:11.869994 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 12:02:11.869994 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 12:02:11.869994 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 12:02:11.869994 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:02:11.869994 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 12:02:11.869994 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:02:11.869994 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:11.869994 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:11.869994 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:11.869994 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:11.869994 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:11.871026 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:11.871026 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:11.871026 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:11.871026 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:11.871026 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:11.871026 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:11.871026 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 12:02:11.871026 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 12:02:11.871026 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 12:02:11.871026 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 12:02:11.871026 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:11.871026 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:11.871026 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:11.871026 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:11.871026 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:11.871026 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:11.872043 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:11.872043 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:11.872043 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:11.872043 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:11.872043 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:11.872043 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:11.872043 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:11.872043 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:11.872043 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:11.872043 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:11.872043 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 12:02:11.872043 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 12:02:11.872043 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 12:02:11.872043 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:02:11.875328 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 454/923] 10.222.0.61 () {56 vars in 1126 bytes} [Fri Aug 26 12:02:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:02:11.877606 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8854118-53ad-4566-b5bd-10d83a6654b9 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:11.920806 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:11.967176 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806fff32-6a48-4f12-83f3-a19be9008bbb None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:11.987933 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:11.998845 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8854118-53ad-4566-b5bd-10d83a6654b9 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:12.017136 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b46dfcb-e442-4055-a0dc-3db54dae8618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:12.018172 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b46dfcb-e442-4055-a0dc-3db54dae8618 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:12.018465 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b46dfcb-e442-4055-a0dc-3db54dae8618 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:12.018742 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b46dfcb-e442-4055-a0dc-3db54dae8618 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:12.019375 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b46dfcb-e442-4055-a0dc-3db54dae8618 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:12.021274 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b46dfcb-e442-4055-a0dc-3db54dae8618 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:12.031890 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-6b46dfcb-e442-4055-a0dc-3db54dae8618 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 12:02:12.031890 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:12.031890 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 12:02:12.031890 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 12:02:12.031890 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 12:02:12.031890 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 12:02:12.031890 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 12:02:12.031890 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 12:02:12.031890 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 12:02:12.031890 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:12.031890 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 12:02:12.031890 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:12.031890 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:12.031890 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:12.031890 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:12.031890 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:12.032889 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:12.032889 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:12.032889 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:12.032889 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:12.032889 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:12.032889 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:12.032889 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:12.032889 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 12:02:12.032889 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 12:02:12.032889 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 12:02:12.032889 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 12:02:12.032889 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:12.032889 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:12.032889 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:12.032889 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:12.032889 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:12.032889 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:12.033778 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:12.033778 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:12.033778 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:12.033778 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:12.033778 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:12.033778 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:12.033778 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:12.033778 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:12.033778 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:12.033778 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:12.033778 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 12:02:12.033778 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 12:02:12.033778 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 26 12:02:12.033778 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:12.033778 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 470/924] 10.222.0.61 () {56 vars in 1132 bytes} [Fri Aug 26 12:02:11 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 12:02:12.038705 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b46dfcb-e442-4055-a0dc-3db54dae8618 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:12.042122 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f4c00c-d4f5-4b9a-a717-cf17e16891d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:12.042732 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4c00c-d4f5-4b9a-a717-cf17e16891d4 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:12.043344 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4c00c-d4f5-4b9a-a717-cf17e16891d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:12.044142 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4c00c-d4f5-4b9a-a717-cf17e16891d4 None admin] PATH_INFO: `/v3/services` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:12.045757 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f4c00c-d4f5-4b9a-a717-cf17e16891d4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:12.046066 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f4c00c-d4f5-4b9a-a717-cf17e16891d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:12.047482 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f4c00c-d4f5-4b9a-a717-cf17e16891d4 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:12.054219 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 455/925] 10.222.0.61 () {56 vars in 1122 bytes} [Fri Aug 26 12:02:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:02:12.060238 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f4c00c-d4f5-4b9a-a717-cf17e16891d4 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:12.153710 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b46dfcb-e442-4055-a0dc-3db54dae8618 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:12.172385 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f4c00c-d4f5-4b9a-a717-cf17e16891d4 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:12.190436 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:12.198005 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf69ee2-ca3f-4d39-a717-114f220a97a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:12.198947 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf69ee2-ca3f-4d39-a717-114f220a97a9 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:12.199464 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf69ee2-ca3f-4d39-a717-114f220a97a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:12.200182 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf69ee2-ca3f-4d39-a717-114f220a97a9 None admin] PATH_INFO: `/v3/services` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:12.201144 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf69ee2-ca3f-4d39-a717-114f220a97a9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:12.201670 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf69ee2-ca3f-4d39-a717-114f220a97a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:12.203462 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf69ee2-ca3f-4d39-a717-114f220a97a9 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:12.213447 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e22f7d7-5511-4742-9412-f06f0a786b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:12.213447 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e22f7d7-5511-4742-9412-f06f0a786b84 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:12.213447 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e22f7d7-5511-4742-9412-f06f0a786b84 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:12.213447 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e22f7d7-5511-4742-9412-f06f0a786b84 None admin] PATH_INFO: `/v3/services` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:12.213844 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 471/926] 10.222.0.61 () {56 vars in 1126 bytes} [Fri Aug 26 12:02:12 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:02:12.213923 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e22f7d7-5511-4742-9412-f06f0a786b84 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:12.214235 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e22f7d7-5511-4742-9412-f06f0a786b84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:12.217676 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e22f7d7-5511-4742-9412-f06f0a786b84 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:12.218711 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf69ee2-ca3f-4d39-a717-114f220a97a9 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:12.226097 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 456/927] 10.222.0.61 () {56 vars in 1122 bytes} [Fri Aug 26 12:02:12 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:12.227386 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e22f7d7-5511-4742-9412-f06f0a786b84 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:12.253332 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:12.348737 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf69ee2-ca3f-4d39-a717-114f220a97a9 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:12.362964 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e22f7d7-5511-4742-9412-f06f0a786b84 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:12.387778 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48424824-d814-44cd-8777-90bc402ad95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:12.388388 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48424824-d814-44cd-8777-90bc402ad95f None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:12.388662 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48424824-d814-44cd-8777-90bc402ad95f None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:12.388985 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48424824-d814-44cd-8777-90bc402ad95f None admin] PATH_INFO: `/v3/services` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:12.389607 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48424824-d814-44cd-8777-90bc402ad95f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:12.389973 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48424824-d814-44cd-8777-90bc402ad95f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:12.391644 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48424824-d814-44cd-8777-90bc402ad95f None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:12.401459 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 472/928] 10.222.0.61 () {56 vars in 1126 bytes} [Fri Aug 26 12:02:12 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:12.402404 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48424824-d814-44cd-8777-90bc402ad95f None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:12.410982 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6432ef53-bcaa-441d-8ed8-41c765385705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:12.411915 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6432ef53-bcaa-441d-8ed8-41c765385705 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:12.412407 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6432ef53-bcaa-441d-8ed8-41c765385705 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:12.412901 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6432ef53-bcaa-441d-8ed8-41c765385705 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:12.413906 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6432ef53-bcaa-441d-8ed8-41c765385705 None admin] RBAC: Adding query filter params (interface=public, service_id=64a8896ee90c4af2a1a20365710abe3a, region_id=RegionOne) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:12.414440 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6432ef53-bcaa-441d-8ed8-41c765385705 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:12.416120 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6432ef53-bcaa-441d-8ed8-41c765385705 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:12.427545 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 457/929] 10.222.0.61 () {56 vars in 1251 bytes} [Fri Aug 26 12:02:12 2022] GET /identity/v3/endpoints?service_id=64a8896ee90c4af2a1a20365710abe3a&interface=public®ion_id=RegionOne => generated 194 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:02:12.428951 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6432ef53-bcaa-441d-8ed8-41c765385705 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:12.465965 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:12.513560 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48424824-d814-44cd-8777-90bc402ad95f None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:12.523736 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:12.542487 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6432ef53-bcaa-441d-8ed8-41c765385705 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:12.552897 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2daf69-e8da-4c60-8dae-ac1dea3c4870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:12.553561 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2daf69-e8da-4c60-8dae-ac1dea3c4870 None admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:12.553839 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2daf69-e8da-4c60-8dae-ac1dea3c4870 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:12.554108 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2daf69-e8da-4c60-8dae-ac1dea3c4870 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:12.554959 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2daf69-e8da-4c60-8dae-ac1dea3c4870 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:12.556471 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2daf69-e8da-4c60-8dae-ac1dea3c4870 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:12.590829 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 473/930] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:02:12 2022] POST /identity/v3/endpoints => generated 347 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 12:02:12.602189 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3e09a8-498b-4f24-81f5-676c365f7bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:12.603118 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e09a8-498b-4f24-81f5-676c365f7bf9 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:12.603668 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e09a8-498b-4f24-81f5-676c365f7bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:12.604087 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e09a8-498b-4f24-81f5-676c365f7bf9 None admin] PATH_INFO: `/v3/services` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:12.604864 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3e09a8-498b-4f24-81f5-676c365f7bf9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:12.607345 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3e09a8-498b-4f24-81f5-676c365f7bf9 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:12.614990 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 458/931] 10.222.0.61 () {56 vars in 1087 bytes} [Fri Aug 26 12:02:12 2022] GET /identity/v3/services => generated 1497 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:02:13.428677 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839de5a4-221f-4dc7-a51e-606209910185 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:13.429413 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839de5a4-221f-4dc7-a51e-606209910185 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:13.429413 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839de5a4-221f-4dc7-a51e-606209910185 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:13.430666 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 474/932] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:02:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:02:13.436701 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4cfb02-f7fc-40db-a83a-f48b58eb8e99 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:13.436701 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4cfb02-f7fc-40db-a83a-f48b58eb8e99 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:13.436701 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4cfb02-f7fc-40db-a83a-f48b58eb8e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:13.455986 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-2e4cfb02-f7fc-40db-a83a-f48b58eb8e99 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:02:13.457490 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-2e4cfb02-f7fc-40db-a83a-f48b58eb8e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:02:13.528763 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 459/933] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:02:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 12:02:13.698971 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11469c05-edeb-4e0d-89e1-e96d433fdf88 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:13.699166 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11469c05-edeb-4e0d-89e1-e96d433fdf88 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:13.699486 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11469c05-edeb-4e0d-89e1-e96d433fdf88 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:13.701586 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 475/934] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:02:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:02:13.707620 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a329ba7-ba84-4233-8a12-f77ef823374b None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:13.708234 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a329ba7-ba84-4233-8a12-f77ef823374b None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:13.708738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a329ba7-ba84-4233-8a12-f77ef823374b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:13.728614 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-7a329ba7-ba84-4233-8a12-f77ef823374b None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:02:13.730203 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-7a329ba7-ba84-4233-8a12-f77ef823374b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:02:13.802484 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 460/935] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:02:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 12:02:13.804165 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11469c05-edeb-4e0d-89e1-e96d433fdf88 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:13.913990 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11469c05-edeb-4e0d-89e1-e96d433fdf88 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:13.928431 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:13.950293 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9159a2-5115-4547-89f4-ec8b48e649a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:13.951396 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9159a2-5115-4547-89f4-ec8b48e649a8 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:13.951874 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9159a2-5115-4547-89f4-ec8b48e649a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:13.952363 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9159a2-5115-4547-89f4-ec8b48e649a8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:13.963582 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9159a2-5115-4547-89f4-ec8b48e649a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:02:13.969031 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-5d9159a2-5115-4547-89f4-ec8b48e649a8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 12:02:13.969031 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:13.969031 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:13.969031 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:13.969031 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:13.969031 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:13.969031 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:13.969031 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:13.969031 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:13.969031 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:13.969031 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:13.969031 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:13.969031 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:02:13.969031 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:02:13.969031 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:02:13.969031 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:02:13.969966 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:02:13.969966 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:02:13.969966 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:02:13.969966 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:02:13.969966 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:13.969966 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:13.969966 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:13.969966 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:13.969966 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:13.969966 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:13.969966 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:13.969966 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:13.969966 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:13.969966 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:13.969966 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:13.969966 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:13.970827 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:13.970827 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:13.970827 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:13.970827 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:13.970827 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:02:13.970827 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:02:13.970827 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:02:13.970827 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:02:13.970827 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:02:13.970827 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:02:13.970827 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 12:02:13.970827 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:13.973009 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 476/936] 10.222.0.61 () {56 vars in 1120 bytes} [Fri Aug 26 12:02:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:02:13.982024 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a329ba7-ba84-4233-8a12-f77ef823374b None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:13.983474 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:14.077256 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9159a2-5115-4547-89f4-ec8b48e649a8 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:14.089781 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a329ba7-ba84-4233-8a12-f77ef823374b None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:14.127630 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf53907d-3f49-499e-b865-b96967369840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:14.128302 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf53907d-3f49-499e-b865-b96967369840 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:14.128594 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf53907d-3f49-499e-b865-b96967369840 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:14.128975 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf53907d-3f49-499e-b865-b96967369840 None admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:14.129757 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf53907d-3f49-499e-b865-b96967369840 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:14.130073 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf53907d-3f49-499e-b865-b96967369840 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:14.131591 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf53907d-3f49-499e-b865-b96967369840 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:14.141617 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 461/937] 10.222.0.61 () {56 vars in 1115 bytes} [Fri Aug 26 12:02:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:14.147189 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf53907d-3f49-499e-b865-b96967369840 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:14.186391 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9159a2-5115-4547-89f4-ec8b48e649a8 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:14.224363 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd5f5c5-e8e4-4485-9431-2b366cd8af97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:14.224878 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd5f5c5-e8e4-4485-9431-2b366cd8af97 None admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:14.225214 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd5f5c5-e8e4-4485-9431-2b366cd8af97 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:14.225484 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd5f5c5-e8e4-4485-9431-2b366cd8af97 None admin] PATH_INFO: `/v3/projects` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:14.226174 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd5f5c5-e8e4-4485-9431-2b366cd8af97 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:14.227658 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd5f5c5-e8e4-4485-9431-2b366cd8af97 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:14.230246 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: WARNING keystone.server.flask.common [None req-ffd5f5c5-e8e4-4485-9431-2b366cd8af97 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 26 12:02:14.236577 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:14.255152 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.sql.core [None req-ffd5f5c5-e8e4-4485-9431-2b366cd8af97 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 26 12:02:14.255152 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 26 12:02:14.255152 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [parameters: {'id': '8bf3913a6f354b82a926682b9416e3ac', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 26 12:02:14.255152 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81712) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 26 12:02:14.263802 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-ffd5f5c5-e8e4-4485-9431-2b366cd8af97 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8bf3913a6f354b82a926682b9416e3ac.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8bf3913a6f354b82a926682b9416e3ac. Aug 26 12:02:14.263802 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:14.263802 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 26 12:02:14.263802 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 26 12:02:14.263802 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 26 12:02:14.263802 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 26 12:02:14.263802 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 26 12:02:14.263802 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application result = self._query(query) Aug 26 12:02:14.263802 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 26 12:02:14.263802 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application conn.query(q) Aug 26 12:02:14.263802 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 26 12:02:14.263802 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 26 12:02:14.263802 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 26 12:02:14.264637 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application result.read() Aug 26 12:02:14.264637 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 26 12:02:14.264637 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 26 12:02:14.264637 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 26 12:02:14.264637 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 26 12:02:14.264637 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 26 12:02:14.264637 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 26 12:02:14.264637 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 26 12:02:14.264637 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 26 12:02:14.264637 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 26 12:02:14.264637 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:14.264637 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 26 12:02:14.264637 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:14.264637 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:14.264637 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 26 12:02:14.264637 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 26 12:02:14.264637 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 26 12:02:14.264637 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 26 12:02:14.265643 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 26 12:02:14.265643 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application next(self.gen) Aug 26 12:02:14.265643 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 26 12:02:14.265643 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application yield resource Aug 26 12:02:14.265643 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 26 12:02:14.265643 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application next(self.gen) Aug 26 12:02:14.265643 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 26 12:02:14.265643 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application self.session.rollback() Aug 26 12:02:14.265643 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 26 12:02:14.265643 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application self.force_reraise() Aug 26 12:02:14.265643 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 26 12:02:14.265643 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise self.value Aug 26 12:02:14.265643 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 26 12:02:14.265643 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 26 12:02:14.265643 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 26 12:02:14.265643 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application session.commit() Aug 26 12:02:14.265643 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 26 12:02:14.266555 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 26 12:02:14.266555 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 26 12:02:14.266555 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application self._prepare_impl() Aug 26 12:02:14.266555 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 26 12:02:14.266555 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application self.session.flush() Aug 26 12:02:14.266555 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 26 12:02:14.266555 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application self._flush(objects) Aug 26 12:02:14.266555 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 26 12:02:14.266555 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 26 12:02:14.266555 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 26 12:02:14.266555 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application compat.raise_( Aug 26 12:02:14.266555 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 26 12:02:14.266555 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception Aug 26 12:02:14.266555 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 26 12:02:14.266555 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application flush_context.execute() Aug 26 12:02:14.266555 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 26 12:02:14.266555 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rec.execute(self) Aug 26 12:02:14.267664 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 26 12:02:14.267664 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 26 12:02:14.267664 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 26 12:02:14.267664 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 26 12:02:14.267664 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 26 12:02:14.267664 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 26 12:02:14.267664 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 26 12:02:14.267664 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 26 12:02:14.267664 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 26 12:02:14.267664 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 26 12:02:14.267664 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 26 12:02:14.267664 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 26 12:02:14.267664 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 26 12:02:14.267664 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 26 12:02:14.267664 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 26 12:02:14.268518 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 26 12:02:14.268518 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 26 12:02:14.268518 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception Aug 26 12:02:14.268518 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 26 12:02:14.268518 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 26 12:02:14.268518 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 26 12:02:14.268518 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 26 12:02:14.268518 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 26 12:02:14.268518 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application result = self._query(query) Aug 26 12:02:14.268518 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 26 12:02:14.268518 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application conn.query(q) Aug 26 12:02:14.268518 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 26 12:02:14.268518 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 26 12:02:14.268518 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 26 12:02:14.268518 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application result.read() Aug 26 12:02:14.268518 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 26 12:02:14.268518 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 26 12:02:14.269384 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 26 12:02:14.269384 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 26 12:02:14.269384 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 26 12:02:14.269384 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 26 12:02:14.269384 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 26 12:02:14.269384 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 26 12:02:14.269384 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 26 12:02:14.269384 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 26 12:02:14.269384 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [parameters: {'id': '8bf3913a6f354b82a926682b9416e3ac', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 26 12:02:14.269384 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 26 12:02:14.269384 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:14.269384 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 12:02:14.269384 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:14.269384 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:14.269384 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 26 12:02:14.270170 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 26 12:02:14.270170 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 26 12:02:14.270170 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 26 12:02:14.270170 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 26 12:02:14.270170 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:14.270170 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 12:02:14.270170 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:14.270170 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:14.270170 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:14.270170 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:14.270170 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:14.270170 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:14.270170 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:14.270170 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:14.270170 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:14.270170 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:14.270170 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:14.271093 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:14.271093 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 26 12:02:14.271093 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 26 12:02:14.271093 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:14.271093 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:14.271093 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 26 12:02:14.271093 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 26 12:02:14.271093 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8bf3913a6f354b82a926682b9416e3ac. Aug 26 12:02:14.271093 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:14.273558 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 477/938] 10.222.0.61 () {60 vars in 1140 bytes} [Fri Aug 26 12:02:14 2022] POST /identity/v3/projects => generated 262 bytes in 197 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 26 12:02:14.280807 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf53907d-3f49-499e-b865-b96967369840 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:14.281377 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd5f5c5-e8e4-4485-9431-2b366cd8af97 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:14.317282 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c154b1-d8aa-4f5e-b582-e0d8bcc7f9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:14.318015 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c154b1-d8aa-4f5e-b582-e0d8bcc7f9dd None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:14.318385 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c154b1-d8aa-4f5e-b582-e0d8bcc7f9dd None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:14.318868 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c154b1-d8aa-4f5e-b582-e0d8bcc7f9dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:14.321697 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:14.331411 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c154b1-d8aa-4f5e-b582-e0d8bcc7f9dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:14.333225 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c154b1-d8aa-4f5e-b582-e0d8bcc7f9dd None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:14.348941 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-04c154b1-d8aa-4f5e-b582-e0d8bcc7f9dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:02:14.348941 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:14.348941 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:14.348941 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:14.348941 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:14.348941 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:14.348941 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:14.348941 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:14.348941 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:14.348941 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:14.348941 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:14.348941 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:14.348941 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:02:14.348941 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:02:14.348941 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:02:14.348941 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:02:14.350200 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:14.350200 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:14.350200 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:14.350200 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:14.350200 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:14.350200 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:14.350200 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:14.350200 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:14.350200 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:14.350200 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:14.350200 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:14.350200 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:14.350200 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:14.350200 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:14.350200 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:14.350200 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:14.350200 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:02:14.351167 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:02:14.351167 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:02:14.351167 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:02:14.351847 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 462/939] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:02:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 12:02:14.356893 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c154b1-d8aa-4f5e-b582-e0d8bcc7f9dd None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:14.408098 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd5f5c5-e8e4-4485-9431-2b366cd8af97 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:14.445046 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e4ce11-598e-4bec-8512-c68d59b743ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:14.446500 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e4ce11-598e-4bec-8512-c68d59b743ea None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:14.446917 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e4ce11-598e-4bec-8512-c68d59b743ea None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:14.447385 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e4ce11-598e-4bec-8512-c68d59b743ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:14.455541 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e4ce11-598e-4bec-8512-c68d59b743ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:14.457521 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e4ce11-598e-4bec-8512-c68d59b743ea None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:14.470526 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-c3e4ce11-598e-4bec-8512-c68d59b743ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:02:14.470526 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:14.470526 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:14.470526 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:14.470526 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:14.470526 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:14.470526 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:14.470526 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:14.470526 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:14.470526 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:14.470526 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:14.470526 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:14.470526 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 12:02:14.470526 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 12:02:14.470526 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 12:02:14.471485 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 12:02:14.471485 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:14.471485 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:14.471485 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:14.471485 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:14.471485 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:14.471485 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:14.471485 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:14.471485 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:14.471485 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:14.471485 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:14.471485 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:14.471485 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:14.471485 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:14.471485 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:14.471485 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:14.471485 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:14.472401 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 12:02:14.472401 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 12:02:14.472401 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 12:02:14.472401 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 12:02:14.472401 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 12:02:14.472401 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 12:02:14.472401 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:02:14.472401 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:14.472855 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c154b1-d8aa-4f5e-b582-e0d8bcc7f9dd None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:14.479811 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 478/940] 10.222.0.61 () {56 vars in 1111 bytes} [Fri Aug 26 12:02:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 12:02:14.480875 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e4ce11-598e-4bec-8512-c68d59b743ea None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:14.490307 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:14.509723 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa09aa8-c011-4712-a44a-1df9d54bbd91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:14.510589 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa09aa8-c011-4712-a44a-1df9d54bbd91 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:14.510906 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa09aa8-c011-4712-a44a-1df9d54bbd91 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:14.511311 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa09aa8-c011-4712-a44a-1df9d54bbd91 None admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:14.512046 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa09aa8-c011-4712-a44a-1df9d54bbd91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:14.513058 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa09aa8-c011-4712-a44a-1df9d54bbd91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:14.514712 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa09aa8-c011-4712-a44a-1df9d54bbd91 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:14.529001 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 463/941] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:02:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:02:14.534036 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa09aa8-c011-4712-a44a-1df9d54bbd91 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:14.592540 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:14.607037 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e4ce11-598e-4bec-8512-c68d59b743ea None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:14.645027 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa09aa8-c011-4712-a44a-1df9d54bbd91 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:14.645251 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b12b5-8191-4004-8ebb-daa38ac8998e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:14.646145 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b12b5-8191-4004-8ebb-daa38ac8998e None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:14.646753 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b12b5-8191-4004-8ebb-daa38ac8998e None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:14.648025 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b12b5-8191-4004-8ebb-daa38ac8998e None admin] PATH_INFO: `/v3/projects` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:14.648890 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8b12b5-8191-4004-8ebb-daa38ac8998e None admin] RBAC: Adding query filter params (name=service) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:14.649404 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8b12b5-8191-4004-8ebb-daa38ac8998e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:14.651061 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8b12b5-8191-4004-8ebb-daa38ac8998e None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:14.676856 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 479/942] 10.222.0.61 () {56 vars in 1112 bytes} [Fri Aug 26 12:02:14 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:14.678153 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6131a-f580-4df8-b298-3334d1fd1d90 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:14.678437 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6131a-f580-4df8-b298-3334d1fd1d90 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:14.678709 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6131a-f580-4df8-b298-3334d1fd1d90 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:14.679975 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 480/943] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:02:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:02:14.681007 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abf7443-79d7-42f0-8a16-b1548c07a095 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:14.681279 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abf7443-79d7-42f0-8a16-b1548c07a095 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:14.681545 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abf7443-79d7-42f0-8a16-b1548c07a095 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:14.683015 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 481/944] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:02:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:02:14.686693 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1107cacf-b3fb-430d-af6f-f07e6f40523e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:14.687923 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1107cacf-b3fb-430d-af6f-f07e6f40523e None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:14.688206 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1107cacf-b3fb-430d-af6f-f07e6f40523e None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:14.688474 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1107cacf-b3fb-430d-af6f-f07e6f40523e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:14.689028 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1cdb89-ad7b-43b5-8453-a6299245498a None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:14.689498 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1cdb89-ad7b-43b5-8453-a6299245498a None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:14.689776 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1cdb89-ad7b-43b5-8453-a6299245498a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:14.697044 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1107cacf-b3fb-430d-af6f-f07e6f40523e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:14.698502 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1107cacf-b3fb-430d-af6f-f07e6f40523e None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:14.709908 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-8f1cdb89-ad7b-43b5-8453-a6299245498a None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:02:14.711308 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-1107cacf-b3fb-430d-af6f-f07e6f40523e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 12:02:14.711308 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:14.711308 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:14.711308 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:14.711308 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:14.711308 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:14.711308 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:14.711308 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:14.711308 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:14.711308 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:14.711308 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:14.711308 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:14.711308 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 12:02:14.711308 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 12:02:14.711308 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 12:02:14.711308 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 12:02:14.712231 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:14.712231 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:14.712231 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 12:02:14.712231 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:02:14.712231 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 12:02:14.712231 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:02:14.712231 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:14.712231 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:14.712231 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:14.712231 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:14.712231 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:14.712231 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:14.712231 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:14.712231 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:14.712231 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:14.712231 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:14.712231 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:14.712231 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:14.713190 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:14.713190 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:14.713190 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 12:02:14.713190 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 12:02:14.713190 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 12:02:14.713190 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 12:02:14.713190 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 12:02:14.713190 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 12:02:14.713190 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 12:02:14.713190 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:02:14.713793 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-8f1cdb89-ad7b-43b5-8453-a6299245498a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:02:14.713866 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 464/945] 10.222.0.61 () {56 vars in 1117 bytes} [Fri Aug 26 12:02:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:02:14.714189 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e07951-f3a2-4271-a279-13e86a8ccd7b None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:14.714473 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e07951-f3a2-4271-a279-13e86a8ccd7b None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:14.714738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e07951-f3a2-4271-a279-13e86a8ccd7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:14.736561 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-c2e07951-f3a2-4271-a279-13e86a8ccd7b None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:02:14.738134 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-c2e07951-f3a2-4271-a279-13e86a8ccd7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:02:14.777668 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:14.801448 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 482/946] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:02:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 12:02:14.802572 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1cdb89-ad7b-43b5-8453-a6299245498a None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:14.834436 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 465/947] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:02:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 12:02:14.902583 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:14.924720 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1cdb89-ad7b-43b5-8453-a6299245498a None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:14.959742 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73049a3-8dcf-49f3-86a2-ee6c3ebc4872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:14.960347 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73049a3-8dcf-49f3-86a2-ee6c3ebc4872 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:14.960625 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73049a3-8dcf-49f3-86a2-ee6c3ebc4872 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:14.960895 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73049a3-8dcf-49f3-86a2-ee6c3ebc4872 None admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:14.961624 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73049a3-8dcf-49f3-86a2-ee6c3ebc4872 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:14.961924 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73049a3-8dcf-49f3-86a2-ee6c3ebc4872 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:14.963374 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73049a3-8dcf-49f3-86a2-ee6c3ebc4872 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:14.982896 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 483/948] 10.222.0.61 () {56 vars in 1149 bytes} [Fri Aug 26 12:02:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:14.988715 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e07951-f3a2-4271-a279-13e86a8ccd7b None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:15.086478 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73049a3-8dcf-49f3-86a2-ee6c3ebc4872 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:15.091274 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e07951-f3a2-4271-a279-13e86a8ccd7b None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:15.137990 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c459e5a4-1d15-4aae-a54a-a3e88dc277f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:15.138908 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c459e5a4-1d15-4aae-a54a-a3e88dc277f8 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:15.139232 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c459e5a4-1d15-4aae-a54a-a3e88dc277f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:15.139680 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c459e5a4-1d15-4aae-a54a-a3e88dc277f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:15.151833 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c459e5a4-1d15-4aae-a54a-a3e88dc277f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:15.153571 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c459e5a4-1d15-4aae-a54a-a3e88dc277f8 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:15.167981 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-c459e5a4-1d15-4aae-a54a-a3e88dc277f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:02:15.167981 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:15.167981 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:15.167981 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:15.167981 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:15.167981 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:15.167981 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:15.167981 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:15.167981 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:15.167981 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:15.167981 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:15.167981 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:15.167981 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:02:15.167981 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:02:15.167981 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:02:15.167981 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:02:15.169660 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:15.169660 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:15.169660 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:15.169660 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:15.169660 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:15.169660 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:15.169660 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:15.169660 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:15.169660 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:15.169660 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:15.169660 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:15.169660 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:15.169660 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:15.169660 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:15.169660 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:15.169660 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:15.169660 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:02:15.170565 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:02:15.170565 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:02:15.170565 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:02:15.170565 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 466/949] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:02:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:02:15.170565 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c459e5a4-1d15-4aae-a54a-a3e88dc277f8 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:15.200404 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:15.203458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73049a3-8dcf-49f3-86a2-ee6c3ebc4872 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:15.240484 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d205be-bb78-445a-adb4-05a32973b352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:15.241127 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d205be-bb78-445a-adb4-05a32973b352 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:15.241414 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d205be-bb78-445a-adb4-05a32973b352 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:15.241721 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d205be-bb78-445a-adb4-05a32973b352 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:15.242309 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d205be-bb78-445a-adb4-05a32973b352 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:15.243717 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d205be-bb78-445a-adb4-05a32973b352 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:15.253259 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-14d205be-bb78-445a-adb4-05a32973b352 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 12:02:15.253259 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:15.253259 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 12:02:15.253259 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 12:02:15.253259 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 12:02:15.253259 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 12:02:15.253259 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 12:02:15.253259 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 12:02:15.253259 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 12:02:15.253259 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:15.253259 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 12:02:15.253259 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:15.253259 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:15.253259 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:15.253259 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:15.253259 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:15.253259 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:15.254457 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:15.254457 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:15.254457 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:15.254457 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:15.254457 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:15.254457 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:15.254457 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 12:02:15.254457 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 12:02:15.254457 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 12:02:15.254457 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 12:02:15.254457 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:15.254457 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:15.254457 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:15.254457 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:15.254457 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:15.254457 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:15.255884 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:15.255884 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:15.255884 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:15.255884 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:15.255884 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:15.255884 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:15.255884 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:15.255884 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:15.255884 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:15.255884 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:15.255884 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 12:02:15.255884 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 12:02:15.255884 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 12:02:15.255884 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:15.259737 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 484/950] 10.222.0.61 () {56 vars in 1126 bytes} [Fri Aug 26 12:02:15 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 12:02:15.260362 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d205be-bb78-445a-adb4-05a32973b352 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:15.264361 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:15.296213 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c459e5a4-1d15-4aae-a54a-a3e88dc277f8 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:15.332876 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1f1f3b-71ee-4d2a-af38-1b4cd0384281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:15.333504 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f1f3b-71ee-4d2a-af38-1b4cd0384281 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:15.333785 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f1f3b-71ee-4d2a-af38-1b4cd0384281 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:15.334079 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f1f3b-71ee-4d2a-af38-1b4cd0384281 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:15.334638 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1f1f3b-71ee-4d2a-af38-1b4cd0384281 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:15.336032 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1f1f3b-71ee-4d2a-af38-1b4cd0384281 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:15.346478 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-4e1f1f3b-71ee-4d2a-af38-1b4cd0384281 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 12:02:15.346478 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:15.346478 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 12:02:15.346478 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 12:02:15.346478 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 12:02:15.346478 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 12:02:15.346478 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 12:02:15.346478 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 12:02:15.346478 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 12:02:15.346478 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:02:15.346478 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 12:02:15.346478 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:02:15.346478 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:15.346478 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:15.346478 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:15.346478 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:15.346478 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:15.347610 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:15.347610 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:15.347610 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:15.347610 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:15.347610 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:15.347610 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:15.347610 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 12:02:15.347610 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 12:02:15.347610 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 12:02:15.347610 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 12:02:15.347610 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:15.347610 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:15.347610 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:15.347610 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:15.347610 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:15.347610 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:15.348581 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:15.348581 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:15.348581 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:15.348581 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:15.348581 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:15.348581 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:15.348581 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:15.348581 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:15.348581 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:15.348581 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:15.348581 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 12:02:15.348581 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 12:02:15.348581 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 12:02:15.348581 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:02:15.348581 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 467/951] 10.222.0.61 () {56 vars in 1111 bytes} [Fri Aug 26 12:02:15 2022] GET /identity/v3/services/compute => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:02:15.349388 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1f1f3b-71ee-4d2a-af38-1b4cd0384281 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:15.377480 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d205be-bb78-445a-adb4-05a32973b352 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:15.413384 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097b8898-a893-49f2-835b-bb584926c845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:15.414008 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097b8898-a893-49f2-835b-bb584926c845 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:15.414283 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097b8898-a893-49f2-835b-bb584926c845 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:15.414550 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097b8898-a893-49f2-835b-bb584926c845 None admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:15.415183 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097b8898-a893-49f2-835b-bb584926c845 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:15.415544 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097b8898-a893-49f2-835b-bb584926c845 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:15.416909 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097b8898-a893-49f2-835b-bb584926c845 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:15.434126 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 485/952] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:02:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:15.434126 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097b8898-a893-49f2-835b-bb584926c845 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:15.447761 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:15.468924 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1f1f3b-71ee-4d2a-af38-1b4cd0384281 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:15.501171 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:15.507027 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5544706e-586d-4831-b2f3-2a789b0f8d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:15.508391 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5544706e-586d-4831-b2f3-2a789b0f8d37 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:15.508683 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5544706e-586d-4831-b2f3-2a789b0f8d37 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:15.508957 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5544706e-586d-4831-b2f3-2a789b0f8d37 None admin] PATH_INFO: `/v3/services` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:15.509587 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5544706e-586d-4831-b2f3-2a789b0f8d37 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:15.509893 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5544706e-586d-4831-b2f3-2a789b0f8d37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:15.511274 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5544706e-586d-4831-b2f3-2a789b0f8d37 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:15.517542 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 468/953] 10.222.0.61 () {56 vars in 1122 bytes} [Fri Aug 26 12:02:15 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:15.518440 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5544706e-586d-4831-b2f3-2a789b0f8d37 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:15.548550 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097b8898-a893-49f2-835b-bb584926c845 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:15.585310 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf81120-ff6a-4caa-a494-9e166a207096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:15.585900 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf81120-ff6a-4caa-a494-9e166a207096 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:15.586180 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf81120-ff6a-4caa-a494-9e166a207096 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:15.586455 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf81120-ff6a-4caa-a494-9e166a207096 None admin] PATH_INFO: `/v3/services` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:15.587078 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf81120-ff6a-4caa-a494-9e166a207096 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:15.587405 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf81120-ff6a-4caa-a494-9e166a207096 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:15.588764 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf81120-ff6a-4caa-a494-9e166a207096 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:15.594224 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 486/954] 10.222.0.61 () {56 vars in 1112 bytes} [Fri Aug 26 12:02:15 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:15.596092 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf81120-ff6a-4caa-a494-9e166a207096 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:15.625545 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5544706e-586d-4831-b2f3-2a789b0f8d37 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:15.661023 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45e23e9-ae10-4728-8c4b-76827dc33c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:15.661645 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e23e9-ae10-4728-8c4b-76827dc33c1b None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:15.661923 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e23e9-ae10-4728-8c4b-76827dc33c1b None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:15.662192 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e23e9-ae10-4728-8c4b-76827dc33c1b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:15.669646 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45e23e9-ae10-4728-8c4b-76827dc33c1b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:15.671093 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45e23e9-ae10-4728-8c4b-76827dc33c1b None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:15.682172 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-c45e23e9-ae10-4728-8c4b-76827dc33c1b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:02:15.682172 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:15.682172 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:15.682172 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:15.682172 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:15.682172 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:15.682172 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:15.682172 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:15.682172 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:15.682172 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:15.682172 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:15.682172 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:15.682172 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 12:02:15.682172 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 12:02:15.682172 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 12:02:15.683081 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 12:02:15.683081 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:15.683081 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:15.683081 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:15.683081 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:15.683081 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:15.683081 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:15.683081 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:15.683081 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:15.683081 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:15.683081 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:15.683081 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:15.683081 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:15.683081 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:15.683081 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:15.683081 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:15.683081 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:15.685300 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 12:02:15.685300 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 12:02:15.685300 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 12:02:15.685300 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 12:02:15.685300 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 12:02:15.685300 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 12:02:15.685300 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:02:15.685300 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:02:15.685300 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 469/955] 10.222.0.61 () {56 vars in 1111 bytes} [Fri Aug 26 12:02:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:02:15.685300 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45e23e9-ae10-4728-8c4b-76827dc33c1b None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:15.699096 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:15.707891 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf81120-ff6a-4caa-a494-9e166a207096 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:15.734952 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:15.752017 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a4b0b-45fa-4e34-bbd2-ed9092211ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:15.752722 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a4b0b-45fa-4e34-bbd2-ed9092211ad4 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:15.752918 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a4b0b-45fa-4e34-bbd2-ed9092211ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:15.753190 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798a4b0b-45fa-4e34-bbd2-ed9092211ad4 None admin] PATH_INFO: `/v3/services` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:15.754125 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798a4b0b-45fa-4e34-bbd2-ed9092211ad4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:15.754242 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798a4b0b-45fa-4e34-bbd2-ed9092211ad4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:15.755598 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798a4b0b-45fa-4e34-bbd2-ed9092211ad4 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:15.766552 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 487/956] 10.222.0.61 () {56 vars in 1122 bytes} [Fri Aug 26 12:02:15 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:15.766552 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a4b0b-45fa-4e34-bbd2-ed9092211ad4 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:15.802271 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45e23e9-ae10-4728-8c4b-76827dc33c1b None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:15.839611 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff9955-028b-4317-9182-4f94baadead0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:15.840616 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff9955-028b-4317-9182-4f94baadead0 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:15.841166 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff9955-028b-4317-9182-4f94baadead0 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:15.841715 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff9955-028b-4317-9182-4f94baadead0 None admin] PATH_INFO: `/v3/services` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:15.842410 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aff9955-028b-4317-9182-4f94baadead0 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:15.842716 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aff9955-028b-4317-9182-4f94baadead0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:15.844230 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aff9955-028b-4317-9182-4f94baadead0 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:15.849407 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 470/957] 10.222.0.61 () {56 vars in 1112 bytes} [Fri Aug 26 12:02:15 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:15.850250 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff9955-028b-4317-9182-4f94baadead0 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:15.878143 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798a4b0b-45fa-4e34-bbd2-ed9092211ad4 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:15.932724 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e27aa-a80f-47cd-8e79-f7f2582344e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:15.934124 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e27aa-a80f-47cd-8e79-f7f2582344e1 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:15.935460 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e27aa-a80f-47cd-8e79-f7f2582344e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:15.936235 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e27aa-a80f-47cd-8e79-f7f2582344e1 None admin] PATH_INFO: `/v3/projects` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:15.938080 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:15.939190 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669e27aa-a80f-47cd-8e79-f7f2582344e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:15.939756 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669e27aa-a80f-47cd-8e79-f7f2582344e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:15.941490 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669e27aa-a80f-47cd-8e79-f7f2582344e1 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:15.959386 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 488/958] 10.222.0.61 () {56 vars in 1148 bytes} [Fri Aug 26 12:02:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:15.963504 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e27aa-a80f-47cd-8e79-f7f2582344e1 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:15.983291 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff9955-028b-4317-9182-4f94baadead0 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:16.029390 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcbf46d-05b8-46c6-94d8-2559bdcce6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:16.029911 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcbf46d-05b8-46c6-94d8-2559bdcce6a4 None admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:16.030191 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcbf46d-05b8-46c6-94d8-2559bdcce6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:16.030453 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcbf46d-05b8-46c6-94d8-2559bdcce6a4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:16.031113 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcbf46d-05b8-46c6-94d8-2559bdcce6a4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:16.032505 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcbf46d-05b8-46c6-94d8-2559bdcce6a4 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:16.055586 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:16.059777 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 471/959] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:02:15 2022] POST /identity/v3/endpoints => generated 349 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:02:16.098456 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e27aa-a80f-47cd-8e79-f7f2582344e1 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:16.148854 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52eb96a-aab7-416b-a5df-0127feeea61a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:16.149694 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52eb96a-aab7-416b-a5df-0127feeea61a None admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:16.150073 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52eb96a-aab7-416b-a5df-0127feeea61a None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:16.150443 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52eb96a-aab7-416b-a5df-0127feeea61a None admin] PATH_INFO: `/v3/projects/ecd68423a83a427681d44dbf266fce6e/users/755b04cae45746569bfbf92f2f213a58/roles/ebae7fe0ca914cd1a0106219ed37c042` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:16.153162 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52eb96a-aab7-416b-a5df-0127feeea61a None admin] RBAC: Authorizing `identity:create_grant(project_id=ecd68423a83a427681d44dbf266fce6e, user_id=755b04cae45746569bfbf92f2f213a58, role_id=ebae7fe0ca914cd1a0106219ed37c042)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:16.154939 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52eb96a-aab7-416b-a5df-0127feeea61a None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:16.174811 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 489/960] 10.222.0.61 () {58 vars in 1439 bytes} [Fri Aug 26 12:02:15 2022] PUT /identity/v3/projects/ecd68423a83a427681d44dbf266fce6e/users/755b04cae45746569bfbf92f2f213a58/roles/ebae7fe0ca914cd1a0106219ed37c042 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:02:16.769027 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848bbbfe-b55b-49b4-81da-c7e33cfe309e None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:16.769506 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848bbbfe-b55b-49b4-81da-c7e33cfe309e None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:16.769581 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848bbbfe-b55b-49b4-81da-c7e33cfe309e None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:16.771121 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 472/961] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:02:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:02:16.776484 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14078be-3141-4666-8757-248c98fceee1 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:16.776905 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14078be-3141-4666-8757-248c98fceee1 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:16.777012 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14078be-3141-4666-8757-248c98fceee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:16.796286 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-a14078be-3141-4666-8757-248c98fceee1 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:02:16.797460 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-a14078be-3141-4666-8757-248c98fceee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:02:16.865386 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 490/962] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:02:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 12:02:17.146695 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848bbbfe-b55b-49b4-81da-c7e33cfe309e None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:17.255443 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848bbbfe-b55b-49b4-81da-c7e33cfe309e None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:17.283137 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:17.291563 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709672f3-3bbf-418b-91a2-6ba893225605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:17.292169 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709672f3-3bbf-418b-91a2-6ba893225605 None admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:17.292450 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709672f3-3bbf-418b-91a2-6ba893225605 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:17.292725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709672f3-3bbf-418b-91a2-6ba893225605 None admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:17.294364 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709672f3-3bbf-418b-91a2-6ba893225605 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:17.296286 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709672f3-3bbf-418b-91a2-6ba893225605 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:17.298905 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: WARNING keystone.server.flask.common [None req-709672f3-3bbf-418b-91a2-6ba893225605 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 26 12:02:17.315918 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.sql.core [None req-709672f3-3bbf-418b-91a2-6ba893225605 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 26 12:02:17.315918 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 26 12:02:17.315918 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [parameters: {'user_id': 'd9843b74f9824658be2cb2dfbbbe0c9d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 26 12:02:17.315918 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81713) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 26 12:02:17.333763 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-709672f3-3bbf-418b-91a2-6ba893225605 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 26 12:02:17.333763 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:17.333763 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 26 12:02:17.333763 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 26 12:02:17.333763 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 26 12:02:17.333763 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 26 12:02:17.333763 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 26 12:02:17.333763 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application result = self._query(query) Aug 26 12:02:17.333763 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 26 12:02:17.333763 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application conn.query(q) Aug 26 12:02:17.333763 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 26 12:02:17.333763 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 26 12:02:17.333763 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 26 12:02:17.333763 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application result.read() Aug 26 12:02:17.333763 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 26 12:02:17.334747 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 26 12:02:17.334747 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 26 12:02:17.334747 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 26 12:02:17.334747 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 26 12:02:17.334747 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 26 12:02:17.334747 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 26 12:02:17.334747 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 26 12:02:17.334747 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 26 12:02:17.334747 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:02:17.334747 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 26 12:02:17.334747 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:02:17.334747 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:17.334747 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 26 12:02:17.334747 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 26 12:02:17.334747 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 26 12:02:17.334747 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 26 12:02:17.334747 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 26 12:02:17.334747 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application next(self.gen) Aug 26 12:02:17.335754 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 26 12:02:17.335754 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application yield resource Aug 26 12:02:17.335754 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 26 12:02:17.335754 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application next(self.gen) Aug 26 12:02:17.335754 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 26 12:02:17.335754 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application self.session.rollback() Aug 26 12:02:17.335754 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 26 12:02:17.335754 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application self.force_reraise() Aug 26 12:02:17.335754 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 26 12:02:17.335754 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise self.value Aug 26 12:02:17.335754 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 26 12:02:17.335754 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 26 12:02:17.335754 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 26 12:02:17.335754 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application session.commit() Aug 26 12:02:17.335754 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 26 12:02:17.335754 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 26 12:02:17.335754 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 26 12:02:17.336673 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application self._prepare_impl() Aug 26 12:02:17.336673 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 26 12:02:17.336673 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application self.session.flush() Aug 26 12:02:17.336673 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 26 12:02:17.336673 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application self._flush(objects) Aug 26 12:02:17.336673 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 26 12:02:17.336673 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 26 12:02:17.336673 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 26 12:02:17.336673 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application compat.raise_( Aug 26 12:02:17.336673 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 26 12:02:17.336673 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception Aug 26 12:02:17.336673 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 26 12:02:17.336673 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application flush_context.execute() Aug 26 12:02:17.336673 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 26 12:02:17.336673 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rec.execute(self) Aug 26 12:02:17.336673 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 26 12:02:17.336673 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 26 12:02:17.337583 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 26 12:02:17.337583 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 26 12:02:17.337583 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 26 12:02:17.337583 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 26 12:02:17.337583 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 26 12:02:17.337583 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 26 12:02:17.337583 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 26 12:02:17.337583 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 26 12:02:17.337583 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 26 12:02:17.337583 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 26 12:02:17.337583 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 26 12:02:17.337583 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 26 12:02:17.337583 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 26 12:02:17.337583 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 26 12:02:17.337583 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 26 12:02:17.337583 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception Aug 26 12:02:17.338457 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 26 12:02:17.338457 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 26 12:02:17.338457 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 26 12:02:17.338457 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 26 12:02:17.338457 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 26 12:02:17.338457 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application result = self._query(query) Aug 26 12:02:17.338457 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 26 12:02:17.338457 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application conn.query(q) Aug 26 12:02:17.338457 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 26 12:02:17.338457 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 26 12:02:17.338457 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 26 12:02:17.338457 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application result.read() Aug 26 12:02:17.338457 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 26 12:02:17.338457 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 26 12:02:17.338457 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 26 12:02:17.338457 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 26 12:02:17.339344 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 26 12:02:17.339344 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 26 12:02:17.339344 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 26 12:02:17.339344 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 26 12:02:17.339344 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 26 12:02:17.339344 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 26 12:02:17.339344 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [parameters: {'user_id': 'd9843b74f9824658be2cb2dfbbbe0c9d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 26 12:02:17.339344 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 26 12:02:17.339344 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:02:17.339344 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 12:02:17.339344 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:02:17.339344 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:17.339344 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:17.339344 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:17.339344 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:17.340862 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:17.340862 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:17.340862 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:17.340862 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:17.340862 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:17.340862 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:17.340862 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:17.340862 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 26 12:02:17.340862 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 26 12:02:17.340862 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:17.340862 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:17.340862 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 12:02:17.340862 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:02:17.340862 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 12:02:17.340862 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:02:17.340862 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 26 12:02:17.340862 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 26 12:02:17.341878 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 26 12:02:17.341878 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 26 12:02:17.341878 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 26 12:02:17.341878 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 26 12:02:17.341878 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 26 12:02:17.341878 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:02:17.341878 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 473/963] 10.222.0.61 () {60 vars in 1131 bytes} [Fri Aug 26 12:02:17 2022] POST /identity/v3/users => generated 160 bytes in 193 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:02:17.343667 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14078be-3141-4666-8757-248c98fceee1 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:17.379263 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:17.449419 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14078be-3141-4666-8757-248c98fceee1 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:17.490940 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f367e26-d831-4dcc-9c3b-1be2b0ac4c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:17.491656 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f367e26-d831-4dcc-9c3b-1be2b0ac4c7b None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:17.491943 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f367e26-d831-4dcc-9c3b-1be2b0ac4c7b None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:17.492213 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f367e26-d831-4dcc-9c3b-1be2b0ac4c7b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:17.499771 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f367e26-d831-4dcc-9c3b-1be2b0ac4c7b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:17.501241 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f367e26-d831-4dcc-9c3b-1be2b0ac4c7b None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:17.513990 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-9f367e26-d831-4dcc-9c3b-1be2b0ac4c7b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 26 12:02:17.513990 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:17.513990 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:17.513990 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:17.513990 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:17.513990 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:17.513990 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:17.513990 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:17.513990 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:17.513990 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:17.513990 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:17.513990 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:17.513990 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 12:02:17.513990 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 12:02:17.513990 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 12:02:17.513990 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 12:02:17.514951 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:17.514951 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:17.514951 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 12:02:17.514951 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:02:17.514951 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 12:02:17.514951 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:02:17.514951 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:17.514951 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:17.514951 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:17.514951 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:17.514951 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:17.514951 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:17.514951 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:17.514951 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:17.514951 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:17.514951 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:17.514951 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:17.514951 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:17.515958 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:17.515958 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:17.515958 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 12:02:17.515958 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 12:02:17.515958 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 12:02:17.515958 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 12:02:17.515958 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 12:02:17.515958 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 12:02:17.515958 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 26 12:02:17.515958 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:17.515958 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 491/964] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:02:17 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:02:17.521426 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709672f3-3bbf-418b-91a2-6ba893225605 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:17.630996 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709672f3-3bbf-418b-91a2-6ba893225605 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:17.668209 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5927b6b-9bc1-49e6-9fc1-a8061926f35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:17.669633 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5927b6b-9bc1-49e6-9fc1-a8061926f35a None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:17.669911 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5927b6b-9bc1-49e6-9fc1-a8061926f35a None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:17.670183 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5927b6b-9bc1-49e6-9fc1-a8061926f35a None admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:17.670882 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5927b6b-9bc1-49e6-9fc1-a8061926f35a None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:17.671184 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5927b6b-9bc1-49e6-9fc1-a8061926f35a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:17.672980 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5927b6b-9bc1-49e6-9fc1-a8061926f35a None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:17.719658 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 474/965] 10.222.0.61 () {56 vars in 1101 bytes} [Fri Aug 26 12:02:17 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:17.899123 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477d08b8-46de-4702-9243-746200bc1b69 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:17.900149 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477d08b8-46de-4702-9243-746200bc1b69 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:17.900635 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477d08b8-46de-4702-9243-746200bc1b69 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:17.902476 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 492/966] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:02:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:02:17.908505 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c149c29-dd41-4a7e-b363-e6e846ac064a None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:17.908971 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c149c29-dd41-4a7e-b363-e6e846ac064a None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:17.909188 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c149c29-dd41-4a7e-b363-e6e846ac064a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:17.929323 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-2c149c29-dd41-4a7e-b363-e6e846ac064a None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:02:17.930842 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-2c149c29-dd41-4a7e-b363-e6e846ac064a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:02:17.982513 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:18.009421 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 475/967] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:02:17 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 12:02:18.063860 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f79a15a-c057-43e5-bc79-75040526c7c8 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:18.064191 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f79a15a-c057-43e5-bc79-75040526c7c8 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:18.064421 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f79a15a-c057-43e5-bc79-75040526c7c8 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:18.065893 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 493/968] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:02:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:02:18.070847 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb8622-8d79-43a7-bc51-0f41100de753 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:18.071135 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb8622-8d79-43a7-bc51-0f41100de753 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:18.071433 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb8622-8d79-43a7-bc51-0f41100de753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:18.089483 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-7cfb8622-8d79-43a7-bc51-0f41100de753 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:02:18.090677 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-7cfb8622-8d79-43a7-bc51-0f41100de753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:02:18.158912 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 476/969] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:02:18 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 12:02:18.194061 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb4abd7-e4c0-449d-ac01-9cd3def17bff None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:18.194348 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb4abd7-e4c0-449d-ac01-9cd3def17bff None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:18.194618 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb4abd7-e4c0-449d-ac01-9cd3def17bff None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:18.196088 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 494/970] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:02:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:02:18.201211 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90416014-d839-4c86-a588-1a02ced11a7a None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:18.201497 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90416014-d839-4c86-a588-1a02ced11a7a None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:18.201767 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90416014-d839-4c86-a588-1a02ced11a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:18.222020 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-90416014-d839-4c86-a588-1a02ced11a7a None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:02:18.223184 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-90416014-d839-4c86-a588-1a02ced11a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:02:18.276384 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb4abd7-e4c0-449d-ac01-9cd3def17bff None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:18.289988 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 477/971] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:02:18 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 12:02:18.376005 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:18.384104 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb4abd7-e4c0-449d-ac01-9cd3def17bff None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:18.418321 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cb2326-a91c-4fbe-b4b6-febb586474f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:18.418914 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb2326-a91c-4fbe-b4b6-febb586474f6 None admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:18.419372 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb2326-a91c-4fbe-b4b6-febb586474f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:18.419646 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb2326-a91c-4fbe-b4b6-febb586474f6 None admin] PATH_INFO: `/v3/services` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:18.420283 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cb2326-a91c-4fbe-b4b6-febb586474f6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:18.421690 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cb2326-a91c-4fbe-b4b6-febb586474f6 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:18.443254 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90416014-d839-4c86-a588-1a02ced11a7a None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:18.443465 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 495/972] 10.222.0.61 () {60 vars in 1141 bytes} [Fri Aug 26 12:02:18 2022] POST /identity/v3/services => generated 242 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 12:02:18.480852 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:18.546958 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90416014-d839-4c86-a588-1a02ced11a7a None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:18.571090 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cb2326-a91c-4fbe-b4b6-febb586474f6 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:18.591190 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f770ca0b-a37f-4918-ba2b-6f69ca040b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:18.591866 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770ca0b-a37f-4918-ba2b-6f69ca040b35 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:18.592147 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770ca0b-a37f-4918-ba2b-6f69ca040b35 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:18.592419 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770ca0b-a37f-4918-ba2b-6f69ca040b35 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:18.593000 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f770ca0b-a37f-4918-ba2b-6f69ca040b35 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:18.594364 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f770ca0b-a37f-4918-ba2b-6f69ca040b35 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:18.603416 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-f770ca0b-a37f-4918-ba2b-6f69ca040b35 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 12:02:18.603416 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:18.603416 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 12:02:18.603416 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 12:02:18.603416 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 12:02:18.603416 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 12:02:18.603416 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 12:02:18.603416 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 12:02:18.603416 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 12:02:18.603416 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:02:18.603416 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 12:02:18.603416 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:02:18.603416 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:18.603416 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:18.603416 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:18.603416 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:18.603416 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:18.604479 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:18.604479 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:18.604479 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:18.604479 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:18.604479 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:18.604479 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:18.604479 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 12:02:18.604479 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 12:02:18.604479 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 12:02:18.604479 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 12:02:18.604479 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:18.604479 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:18.604479 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:18.604479 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:18.604479 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:18.604479 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:18.605356 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:18.605356 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:18.605356 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:18.605356 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:18.605356 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:18.605356 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:18.605356 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:18.605356 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:18.605356 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:18.605356 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:18.605356 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 12:02:18.605356 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 12:02:18.605356 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 12:02:18.605356 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:02:18.605356 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 478/973] 10.222.0.61 () {56 vars in 1126 bytes} [Fri Aug 26 12:02:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 12:02:18.610936 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f770ca0b-a37f-4918-ba2b-6f69ca040b35 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:18.698724 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cb2326-a91c-4fbe-b4b6-febb586474f6 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:18.716496 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:18.719980 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f770ca0b-a37f-4918-ba2b-6f69ca040b35 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:18.738981 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e390efa3-21b8-4b6d-9b5c-a164844b6b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:18.740067 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e390efa3-21b8-4b6d-9b5c-a164844b6b48 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:18.740351 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e390efa3-21b8-4b6d-9b5c-a164844b6b48 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:18.740622 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e390efa3-21b8-4b6d-9b5c-a164844b6b48 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:18.756621 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd118ef1-4a1b-46bd-9151-df8ce35be575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:18.757222 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd118ef1-4a1b-46bd-9151-df8ce35be575 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:18.757497 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd118ef1-4a1b-46bd-9151-df8ce35be575 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:18.757765 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd118ef1-4a1b-46bd-9151-df8ce35be575 None admin] PATH_INFO: `/v3/services` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:18.758636 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd118ef1-4a1b-46bd-9151-df8ce35be575 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:18.758931 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd118ef1-4a1b-46bd-9151-df8ce35be575 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:18.759073 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e390efa3-21b8-4b6d-9b5c-a164844b6b48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:02:18.760329 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd118ef1-4a1b-46bd-9151-df8ce35be575 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:18.764516 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-e390efa3-21b8-4b6d-9b5c-a164844b6b48 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 12:02:18.764516 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:18.764516 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:18.764516 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:18.764516 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:18.764516 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:18.764516 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:18.764516 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:18.764516 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:18.764516 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:18.764516 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:18.764516 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:18.764516 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:02:18.764516 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:02:18.764516 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:02:18.764516 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:02:18.765458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:02:18.765458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:02:18.765458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:02:18.765458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:02:18.765458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:18.765458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:18.765458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:18.765458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:18.765458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:18.765458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:18.765458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:18.765458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:18.765458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:18.765458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:18.765458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:18.765458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:18.766423 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:18.766423 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:18.766423 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:18.766423 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:18.766423 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:02:18.766423 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:02:18.766423 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:02:18.766423 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:02:18.766423 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:02:18.766423 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:02:18.766423 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 12:02:18.766423 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:18.770181 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 479/974] 10.222.0.61 () {56 vars in 1122 bytes} [Fri Aug 26 12:02:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:02:18.770548 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 496/975] 10.222.0.61 () {56 vars in 1120 bytes} [Fri Aug 26 12:02:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 12:02:18.771187 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd118ef1-4a1b-46bd-9151-df8ce35be575 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:18.771307 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e390efa3-21b8-4b6d-9b5c-a164844b6b48 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:18.792189 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:18.880697 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd118ef1-4a1b-46bd-9151-df8ce35be575 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:18.885652 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e390efa3-21b8-4b6d-9b5c-a164844b6b48 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:18.917476 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20912043-256a-4a54-857e-e839a6eab1e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:18.918201 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20912043-256a-4a54-857e-e839a6eab1e0 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:18.919031 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20912043-256a-4a54-857e-e839a6eab1e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:18.919324 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20912043-256a-4a54-857e-e839a6eab1e0 None admin] PATH_INFO: `/v3/services` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:18.920000 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20912043-256a-4a54-857e-e839a6eab1e0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:18.920305 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20912043-256a-4a54-857e-e839a6eab1e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:18.921382 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8149796a-afd9-47d7-9195-39bc461213a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:18.921787 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20912043-256a-4a54-857e-e839a6eab1e0 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:18.921990 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8149796a-afd9-47d7-9195-39bc461213a3 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:18.922272 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8149796a-afd9-47d7-9195-39bc461213a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:18.922548 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8149796a-afd9-47d7-9195-39bc461213a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:18.923271 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8149796a-afd9-47d7-9195-39bc461213a3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:18.923548 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8149796a-afd9-47d7-9195-39bc461213a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:18.924903 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8149796a-afd9-47d7-9195-39bc461213a3 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:18.931825 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 480/976] 10.222.0.61 () {56 vars in 1122 bytes} [Fri Aug 26 12:02:18 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:18.932586 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20912043-256a-4a54-857e-e839a6eab1e0 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:18.939058 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 497/977] 10.222.0.61 () {56 vars in 1115 bytes} [Fri Aug 26 12:02:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:18.939938 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8149796a-afd9-47d7-9195-39bc461213a3 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:18.968674 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:19.027595 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:19.047737 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20912043-256a-4a54-857e-e839a6eab1e0 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:19.050462 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8149796a-afd9-47d7-9195-39bc461213a3 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:19.082964 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef65c105-b03f-4bb1-b927-5e4d4c16a6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:19.083605 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef65c105-b03f-4bb1-b927-5e4d4c16a6db None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:19.083886 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef65c105-b03f-4bb1-b927-5e4d4c16a6db None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:19.084160 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef65c105-b03f-4bb1-b927-5e4d4c16a6db None admin] PATH_INFO: `/v3/endpoints` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:19.084720 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fec41b-59c8-47d7-b16d-fe5f24b91bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:19.084929 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef65c105-b03f-4bb1-b927-5e4d4c16a6db None admin] RBAC: Adding query filter params (interface=admin, service_id=64a8896ee90c4af2a1a20365710abe3a, region_id=RegionOne) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:19.085178 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef65c105-b03f-4bb1-b927-5e4d4c16a6db None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:19.085337 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fec41b-59c8-47d7-b16d-fe5f24b91bc2 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:19.085613 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fec41b-59c8-47d7-b16d-fe5f24b91bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:19.085882 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fec41b-59c8-47d7-b16d-fe5f24b91bc2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:19.086561 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef65c105-b03f-4bb1-b927-5e4d4c16a6db None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:19.096225 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 481/978] 10.222.0.61 () {56 vars in 1249 bytes} [Fri Aug 26 12:02:18 2022] GET /identity/v3/endpoints?service_id=64a8896ee90c4af2a1a20365710abe3a&interface=admin®ion_id=RegionOne => generated 193 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:02:19.097066 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef65c105-b03f-4bb1-b927-5e4d4c16a6db None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:19.100469 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fec41b-59c8-47d7-b16d-fe5f24b91bc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:19.101917 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fec41b-59c8-47d7-b16d-fe5f24b91bc2 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:19.116711 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-41fec41b-59c8-47d7-b16d-fe5f24b91bc2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:02:19.116711 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:19.116711 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:19.116711 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:19.116711 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:19.116711 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:19.116711 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:19.116711 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:19.116711 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:19.116711 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:19.116711 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:19.116711 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:19.116711 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:02:19.116711 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:02:19.116711 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:02:19.116711 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:02:19.117697 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:19.117697 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:19.117697 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:19.117697 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:19.117697 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:19.117697 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:19.117697 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:19.117697 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:19.117697 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:19.117697 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:19.117697 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:19.117697 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:19.117697 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:19.117697 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:19.117697 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:19.117697 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:19.117697 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:02:19.124000 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:02:19.124000 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:02:19.124000 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:19.124000 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 498/979] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:02:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 12:02:19.124000 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fec41b-59c8-47d7-b16d-fe5f24b91bc2 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:19.195413 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:19.208273 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef65c105-b03f-4bb1-b927-5e4d4c16a6db None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:19.235562 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fec41b-59c8-47d7-b16d-fe5f24b91bc2 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:19.244788 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2e8435-1a59-4322-a2b2-03b09bbd62de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:19.245772 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e8435-1a59-4322-a2b2-03b09bbd62de None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:19.246148 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e8435-1a59-4322-a2b2-03b09bbd62de None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:19.246467 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e8435-1a59-4322-a2b2-03b09bbd62de None admin] PATH_INFO: `/v3/services` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:19.247150 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2e8435-1a59-4322-a2b2-03b09bbd62de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:19.248660 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2e8435-1a59-4322-a2b2-03b09bbd62de None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:19.263241 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 482/980] 10.222.0.61 () {56 vars in 1087 bytes} [Fri Aug 26 12:02:19 2022] GET /identity/v3/services => generated 1727 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 12:02:19.264408 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:19.271542 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a444d55f-a9af-4a3f-ad27-fe05474f53d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:19.272309 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a444d55f-a9af-4a3f-ad27-fe05474f53d7 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:19.272657 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a444d55f-a9af-4a3f-ad27-fe05474f53d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:19.273002 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a444d55f-a9af-4a3f-ad27-fe05474f53d7 None admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:19.273789 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a444d55f-a9af-4a3f-ad27-fe05474f53d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:19.274165 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a444d55f-a9af-4a3f-ad27-fe05474f53d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:19.275692 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a444d55f-a9af-4a3f-ad27-fe05474f53d7 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:19.295396 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 499/981] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:02:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:02:19.296759 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2e8435-1a59-4322-a2b2-03b09bbd62de None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:19.407118 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2e8435-1a59-4322-a2b2-03b09bbd62de None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:19.444610 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2581e09f-3101-49a7-add7-88e89ce652ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:19.445237 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2581e09f-3101-49a7-add7-88e89ce652ae None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:19.445569 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2581e09f-3101-49a7-add7-88e89ce652ae None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:19.445783 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2581e09f-3101-49a7-add7-88e89ce652ae None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:19.453783 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2581e09f-3101-49a7-add7-88e89ce652ae None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:19.455239 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2581e09f-3101-49a7-add7-88e89ce652ae None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:19.467503 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-2581e09f-3101-49a7-add7-88e89ce652ae None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 12:02:19.467503 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:19.467503 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:19.467503 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:19.467503 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:19.467503 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:19.467503 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:19.467503 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:19.467503 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:19.467503 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:19.467503 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:19.467503 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:19.467503 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 12:02:19.467503 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 12:02:19.467503 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 12:02:19.467503 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 12:02:19.468498 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:19.468498 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:19.468498 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 12:02:19.468498 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:02:19.468498 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 12:02:19.468498 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:02:19.468498 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:19.468498 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:19.468498 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:19.468498 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:19.468498 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:19.468498 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:19.468498 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:19.468498 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:19.468498 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:19.468498 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:19.468498 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:19.468498 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:19.469451 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:19.469451 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:19.469451 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 12:02:19.469451 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 12:02:19.469451 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 12:02:19.469451 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 12:02:19.469451 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 12:02:19.469451 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 12:02:19.469451 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 26 12:02:19.469451 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:02:19.469451 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 483/982] 10.222.0.61 () {56 vars in 1117 bytes} [Fri Aug 26 12:02:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:02:19.475797 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a444d55f-a9af-4a3f-ad27-fe05474f53d7 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:19.582704 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a444d55f-a9af-4a3f-ad27-fe05474f53d7 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:19.617009 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4855003-7a9e-4fee-b418-fa030b55898c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:19.617612 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4855003-7a9e-4fee-b418-fa030b55898c None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:19.617889 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4855003-7a9e-4fee-b418-fa030b55898c None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:19.618161 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4855003-7a9e-4fee-b418-fa030b55898c None admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:19.618877 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4855003-7a9e-4fee-b418-fa030b55898c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:19.619179 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4855003-7a9e-4fee-b418-fa030b55898c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:19.620617 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4855003-7a9e-4fee-b418-fa030b55898c None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:19.643134 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 500/983] 10.222.0.61 () {56 vars in 1149 bytes} [Fri Aug 26 12:02:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:19.644412 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2581e09f-3101-49a7-add7-88e89ce652ae None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:19.660915 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:19.756444 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2581e09f-3101-49a7-add7-88e89ce652ae None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:19.790934 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742d428f-5af7-4419-926d-8580a3e422a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:19.791576 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742d428f-5af7-4419-926d-8580a3e422a9 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:19.791855 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742d428f-5af7-4419-926d-8580a3e422a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:19.792122 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742d428f-5af7-4419-926d-8580a3e422a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:19.858797 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742d428f-5af7-4419-926d-8580a3e422a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:19.860267 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742d428f-5af7-4419-926d-8580a3e422a9 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:19.875132 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:19.960570 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 484/984] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:02:19 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 12:02:19.967431 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4855003-7a9e-4fee-b418-fa030b55898c None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:20.018883 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:20.071585 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4855003-7a9e-4fee-b418-fa030b55898c None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:20.106643 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcdc821-e799-46af-b2f1-e455bf1c1894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:20.107294 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcdc821-e799-46af-b2f1-e455bf1c1894 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:20.107571 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcdc821-e799-46af-b2f1-e455bf1c1894 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:20.107835 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcdc821-e799-46af-b2f1-e455bf1c1894 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:20.117981 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcdc821-e799-46af-b2f1-e455bf1c1894 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:20.119432 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcdc821-e799-46af-b2f1-e455bf1c1894 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:20.134703 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-2dcdc821-e799-46af-b2f1-e455bf1c1894 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:02:20.134703 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:20.134703 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:20.134703 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:20.134703 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:20.134703 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:20.134703 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:20.134703 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:20.134703 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:20.134703 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:20.134703 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:20.134703 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:20.134703 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:02:20.134703 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:02:20.134703 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:02:20.134703 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:02:20.135904 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:20.135904 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:20.135904 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:20.135904 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:20.135904 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:20.135904 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:20.135904 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:20.135904 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:20.135904 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:20.135904 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:20.135904 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:20.135904 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:20.135904 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:20.135904 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:20.135904 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:20.135904 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:20.135904 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:02:20.136828 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:02:20.136828 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:02:20.136828 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:20.138383 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 501/985] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:02:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 12:02:20.142369 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742d428f-5af7-4419-926d-8580a3e422a9 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:20.244669 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742d428f-5af7-4419-926d-8580a3e422a9 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:20.260011 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:20.281678 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d3f7ec-ee9a-4c22-a460-3f1677a244c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:20.282210 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d3f7ec-ee9a-4c22-a460-3f1677a244c1 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:20.282486 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d3f7ec-ee9a-4c22-a460-3f1677a244c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:20.282756 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d3f7ec-ee9a-4c22-a460-3f1677a244c1 None admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:20.283432 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d3f7ec-ee9a-4c22-a460-3f1677a244c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:20.283738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d3f7ec-ee9a-4c22-a460-3f1677a244c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:20.285108 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d3f7ec-ee9a-4c22-a460-3f1677a244c1 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:20.301472 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 485/986] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:02:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:02:20.302895 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcdc821-e799-46af-b2f1-e455bf1c1894 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:20.406480 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78888c52-4abb-42f7-8503-e61ede8f09fc None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:20.406793 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78888c52-4abb-42f7-8503-e61ede8f09fc None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:20.407048 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78888c52-4abb-42f7-8503-e61ede8f09fc None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:20.408433 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 486/987] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:02:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 12:02:20.408747 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:20.412129 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcdc821-e799-46af-b2f1-e455bf1c1894 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:20.415406 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3364cb63-a6d8-4946-8101-6816243e557c None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:20.415406 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3364cb63-a6d8-4946-8101-6816243e557c None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:20.415618 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3364cb63-a6d8-4946-8101-6816243e557c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:20.437243 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-3364cb63-a6d8-4946-8101-6816243e557c None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:02:20.438444 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-3364cb63-a6d8-4946-8101-6816243e557c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:02:20.463798 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14371c65-de43-4abc-9595-14d01e40aca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:20.464793 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14371c65-de43-4abc-9595-14d01e40aca2 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:20.464793 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14371c65-de43-4abc-9595-14d01e40aca2 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:20.466246 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14371c65-de43-4abc-9595-14d01e40aca2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:20.474517 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14371c65-de43-4abc-9595-14d01e40aca2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:20.476791 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14371c65-de43-4abc-9595-14d01e40aca2 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:20.491071 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-14371c65-de43-4abc-9595-14d01e40aca2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:02:20.491071 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:20.491071 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:20.491071 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:20.491071 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:20.491071 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:20.491071 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:20.491071 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:20.491071 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:20.491071 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:20.491071 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:20.491071 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:20.491071 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 12:02:20.491071 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 12:02:20.491071 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 12:02:20.492025 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 12:02:20.492025 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:20.492025 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:20.492025 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:20.492025 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:20.492025 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:20.492025 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:20.492025 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:20.492025 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:20.492025 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:20.492025 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:20.492025 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:20.492025 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:20.492025 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:20.492025 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:20.492025 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:20.492025 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:20.492939 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 12:02:20.492939 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 12:02:20.492939 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 12:02:20.492939 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 12:02:20.492939 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 12:02:20.492939 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 12:02:20.492939 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:02:20.492939 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:20.492939 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 502/988] 10.222.0.61 () {56 vars in 1111 bytes} [Fri Aug 26 12:02:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 12:02:20.498602 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14371c65-de43-4abc-9595-14d01e40aca2 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:20.533286 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 487/989] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:02:20 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 12:02:20.620126 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14371c65-de43-4abc-9595-14d01e40aca2 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:20.657539 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298965a3-ae0b-4ef8-9e7b-c8fb9f334812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:20.658482 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298965a3-ae0b-4ef8-9e7b-c8fb9f334812 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:20.658943 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298965a3-ae0b-4ef8-9e7b-c8fb9f334812 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:20.659434 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298965a3-ae0b-4ef8-9e7b-c8fb9f334812 None admin] PATH_INFO: `/v3/projects` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:20.660348 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298965a3-ae0b-4ef8-9e7b-c8fb9f334812 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:20.660864 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298965a3-ae0b-4ef8-9e7b-c8fb9f334812 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:20.662522 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298965a3-ae0b-4ef8-9e7b-c8fb9f334812 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:20.676017 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 503/990] 10.222.0.61 () {56 vars in 1148 bytes} [Fri Aug 26 12:02:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:02:20.681961 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3364cb63-a6d8-4946-8101-6816243e557c None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:20.799297 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3364cb63-a6d8-4946-8101-6816243e557c None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:20.801570 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:20.818453 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298965a3-ae0b-4ef8-9e7b-c8fb9f334812 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:20.838636 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586635b2-19fe-4d58-a51d-b196286533ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:20.839268 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586635b2-19fe-4d58-a51d-b196286533ff None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:20.839676 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586635b2-19fe-4d58-a51d-b196286533ff None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:20.839942 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586635b2-19fe-4d58-a51d-b196286533ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:20.841600 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586635b2-19fe-4d58-a51d-b196286533ff None admin] RBAC: Adding query filter params (role.id=ebae7fe0ca914cd1a0106219ed37c042, scope.project.id=ecd68423a83a427681d44dbf266fce6e, user.id=755b04cae45746569bfbf92f2f213a58) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:20.841920 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586635b2-19fe-4d58-a51d-b196286533ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:20.843452 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586635b2-19fe-4d58-a51d-b196286533ff None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:20.844270 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:20.853446 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 488/991] 10.222.0.61 () {56 vars in 1374 bytes} [Fri Aug 26 12:02:20 2022] GET /identity/v3/role_assignments?user.id=755b04cae45746569bfbf92f2f213a58&scope.project.id=ecd68423a83a427681d44dbf266fce6e&role.id=ebae7fe0ca914cd1a0106219ed37c042 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:20.945931 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298965a3-ae0b-4ef8-9e7b-c8fb9f334812 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:20.983000 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65107e1-5e3d-42ab-8114-6909ff0e3554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:20.983801 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65107e1-5e3d-42ab-8114-6909ff0e3554 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:20.983988 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65107e1-5e3d-42ab-8114-6909ff0e3554 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:20.984259 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65107e1-5e3d-42ab-8114-6909ff0e3554 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:20.984856 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65107e1-5e3d-42ab-8114-6909ff0e3554 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:20.986233 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65107e1-5e3d-42ab-8114-6909ff0e3554 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:20.995535 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-d65107e1-5e3d-42ab-8114-6909ff0e3554 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 12:02:20.995535 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:20.995535 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 12:02:20.995535 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 12:02:20.995535 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 12:02:20.995535 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 12:02:20.995535 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 12:02:20.995535 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 12:02:20.995535 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 12:02:20.995535 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:20.995535 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 12:02:20.995535 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:20.995535 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:20.995535 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:20.995535 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:20.995535 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:20.995535 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:20.997265 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:20.997265 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:20.997265 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:20.997265 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:20.997265 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:20.997265 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:20.997265 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 12:02:20.997265 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 12:02:20.997265 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 12:02:20.997265 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 12:02:20.997265 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:20.997265 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:20.997265 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:20.997265 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:20.997265 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:20.997265 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:20.998111 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:20.998111 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:20.998111 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:20.998111 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:20.998111 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:20.998111 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:20.998111 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:20.998111 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:20.998111 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:20.998111 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:20.998111 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 12:02:20.998111 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 12:02:20.998111 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 12:02:20.998111 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:20.998111 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 504/992] 10.222.0.61 () {56 vars in 1111 bytes} [Fri Aug 26 12:02:20 2022] GET /identity/v3/services/compute => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 12:02:21.003371 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586635b2-19fe-4d58-a51d-b196286533ff None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:21.109899 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586635b2-19fe-4d58-a51d-b196286533ff None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:21.144222 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b5a9cd-f7b5-4903-b84b-136712f98174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:21.144884 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5a9cd-f7b5-4903-b84b-136712f98174 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:21.145164 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5a9cd-f7b5-4903-b84b-136712f98174 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:21.145429 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5a9cd-f7b5-4903-b84b-136712f98174 None admin] PATH_INFO: `/v3/services` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:21.146053 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b5a9cd-f7b5-4903-b84b-136712f98174 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:21.146349 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b5a9cd-f7b5-4903-b84b-136712f98174 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:21.147739 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b5a9cd-f7b5-4903-b84b-136712f98174 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:21.157166 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 489/993] 10.222.0.61 () {56 vars in 1112 bytes} [Fri Aug 26 12:02:21 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:21.158687 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65107e1-5e3d-42ab-8114-6909ff0e3554 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:21.185561 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f46b99-e63b-4364-a323-a044e02ee449 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:21.185830 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f46b99-e63b-4364-a323-a044e02ee449 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:21.186103 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f46b99-e63b-4364-a323-a044e02ee449 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:21.187620 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 490/994] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:02:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 12:02:21.192752 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97a06a-053a-4e3a-be51-338937e935a3 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:21.193033 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97a06a-053a-4e3a-be51-338937e935a3 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:21.193305 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97a06a-053a-4e3a-be51-338937e935a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:21.212855 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-dd97a06a-053a-4e3a-be51-338937e935a3 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:02:21.214999 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-dd97a06a-053a-4e3a-be51-338937e935a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:02:21.244417 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:21.253434 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:21.272459 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65107e1-5e3d-42ab-8114-6909ff0e3554 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:21.288511 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 491/995] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:02:21 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 12:02:21.312565 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa15473-f9f5-4aa8-b86c-8662efd58ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:21.313179 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa15473-f9f5-4aa8-b86c-8662efd58ca5 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:21.313515 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa15473-f9f5-4aa8-b86c-8662efd58ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:21.313795 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa15473-f9f5-4aa8-b86c-8662efd58ca5 None admin] PATH_INFO: `/v3/services` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:21.314477 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa15473-f9f5-4aa8-b86c-8662efd58ca5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:21.314784 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa15473-f9f5-4aa8-b86c-8662efd58ca5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:21.316349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa15473-f9f5-4aa8-b86c-8662efd58ca5 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:21.326688 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 505/996] 10.222.0.61 () {56 vars in 1112 bytes} [Fri Aug 26 12:02:21 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:21.327764 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd97a06a-053a-4e3a-be51-338937e935a3 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:21.429511 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd97a06a-053a-4e3a-be51-338937e935a3 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:21.464030 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f0d83f-70fa-4e9a-9f7d-e3f9160fd2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:21.464626 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0d83f-70fa-4e9a-9f7d-e3f9160fd2d8 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:21.464907 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0d83f-70fa-4e9a-9f7d-e3f9160fd2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:21.465220 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0d83f-70fa-4e9a-9f7d-e3f9160fd2d8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:21.466244 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f0d83f-70fa-4e9a-9f7d-e3f9160fd2d8 None admin] RBAC: Adding query filter params (interface=public, service_id=f127cf938ad949559b2ff65417a77252, region_id=RegionOne) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:21.466556 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f0d83f-70fa-4e9a-9f7d-e3f9160fd2d8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:21.467963 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f0d83f-70fa-4e9a-9f7d-e3f9160fd2d8 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:21.477363 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 492/997] 10.222.0.61 () {56 vars in 1251 bytes} [Fri Aug 26 12:02:21 2022] GET /identity/v3/endpoints?service_id=f127cf938ad949559b2ff65417a77252&interface=public®ion_id=RegionOne => generated 194 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:21.478392 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa15473-f9f5-4aa8-b86c-8662efd58ca5 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:21.562212 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f0d83f-70fa-4e9a-9f7d-e3f9160fd2d8 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:21.583236 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa15473-f9f5-4aa8-b86c-8662efd58ca5 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:21.589352 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:21.622048 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230bce47-bf52-42f9-94f4-68b706ffb8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:21.622763 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230bce47-bf52-42f9-94f4-68b706ffb8ed None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:21.623063 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230bce47-bf52-42f9-94f4-68b706ffb8ed None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:21.623568 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230bce47-bf52-42f9-94f4-68b706ffb8ed None admin] PATH_INFO: `/v3/services` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:21.624173 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230bce47-bf52-42f9-94f4-68b706ffb8ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:21.625641 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230bce47-bf52-42f9-94f4-68b706ffb8ed None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:21.643236 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 506/998] 10.222.0.61 () {56 vars in 1087 bytes} [Fri Aug 26 12:02:21 2022] GET /identity/v3/services => generated 1727 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 12:02:21.678954 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f0d83f-70fa-4e9a-9f7d-e3f9160fd2d8 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:21.716607 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c522e23-40fe-419f-b281-76ab3f599ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:21.717161 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c522e23-40fe-419f-b281-76ab3f599ba8 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:21.717441 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c522e23-40fe-419f-b281-76ab3f599ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:21.717715 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c522e23-40fe-419f-b281-76ab3f599ba8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:21.718306 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c522e23-40fe-419f-b281-76ab3f599ba8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:21.719742 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c522e23-40fe-419f-b281-76ab3f599ba8 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:21.729215 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-1c522e23-40fe-419f-b281-76ab3f599ba8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 12:02:21.729215 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:21.729215 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 12:02:21.729215 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 12:02:21.729215 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 12:02:21.729215 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 12:02:21.729215 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 12:02:21.729215 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 12:02:21.729215 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 12:02:21.729215 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:02:21.729215 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 12:02:21.729215 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:02:21.729215 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:21.729215 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:21.729215 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:21.729215 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:21.729215 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:21.730290 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:21.730290 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:21.730290 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:21.730290 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:21.730290 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:21.730290 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:21.730290 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 12:02:21.730290 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 12:02:21.730290 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 12:02:21.730290 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 12:02:21.730290 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:21.730290 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:21.730290 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:21.730290 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:21.730290 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:21.730290 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:21.736138 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:21.736138 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:21.736138 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:21.736138 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:21.736138 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:21.736138 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:21.736138 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:21.736138 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:21.736138 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:21.736138 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:21.736138 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 12:02:21.736138 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 12:02:21.736138 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 26 12:02:21.736138 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:02:21.736138 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 493/999] 10.222.0.61 () {56 vars in 1126 bytes} [Fri Aug 26 12:02:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 12:02:21.738130 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230bce47-bf52-42f9-94f4-68b706ffb8ed None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:21.756079 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:21.851508 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230bce47-bf52-42f9-94f4-68b706ffb8ed None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:21.887141 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72770dd-b85c-4a47-9430-4974b8a444ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:21.887780 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72770dd-b85c-4a47-9430-4974b8a444ed None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:21.888060 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72770dd-b85c-4a47-9430-4974b8a444ed None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:21.888335 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72770dd-b85c-4a47-9430-4974b8a444ed None admin] PATH_INFO: `/v3/services` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:21.888970 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72770dd-b85c-4a47-9430-4974b8a444ed None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:21.889273 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72770dd-b85c-4a47-9430-4974b8a444ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:21.890637 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72770dd-b85c-4a47-9430-4974b8a444ed None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:21.895779 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 507/1000] 10.222.0.61 () {56 vars in 1122 bytes} [Fri Aug 26 12:02:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:21.901650 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c522e23-40fe-419f-b281-76ab3f599ba8 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:22.011353 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:22.022261 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c522e23-40fe-419f-b281-76ab3f599ba8 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:22.056025 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7040811-0c0f-4c96-beca-e4fbb0aa4baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:22.056631 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7040811-0c0f-4c96-beca-e4fbb0aa4baf None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:22.056906 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7040811-0c0f-4c96-beca-e4fbb0aa4baf None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:22.057174 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7040811-0c0f-4c96-beca-e4fbb0aa4baf None admin] PATH_INFO: `/v3/services` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:22.057792 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7040811-0c0f-4c96-beca-e4fbb0aa4baf None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:22.058092 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7040811-0c0f-4c96-beca-e4fbb0aa4baf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:22.060119 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7040811-0c0f-4c96-beca-e4fbb0aa4baf None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:22.066050 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 494/1001] 10.222.0.61 () {56 vars in 1122 bytes} [Fri Aug 26 12:02:21 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:22.070887 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72770dd-b85c-4a47-9430-4974b8a444ed None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:22.152530 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:22.177652 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72770dd-b85c-4a47-9430-4974b8a444ed None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:22.210722 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158e527d-9120-4022-8551-33ff113c62a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:22.211349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e527d-9120-4022-8551-33ff113c62a3 None admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:22.211629 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e527d-9120-4022-8551-33ff113c62a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:22.211901 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e527d-9120-4022-8551-33ff113c62a3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:22.212561 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158e527d-9120-4022-8551-33ff113c62a3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:22.213913 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158e527d-9120-4022-8551-33ff113c62a3 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:22.239261 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 508/1002] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:02:22 2022] POST /identity/v3/endpoints => generated 325 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:02:22.831799 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed5c8e3-db48-40cd-870c-aaae5b014673 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:22.832149 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed5c8e3-db48-40cd-870c-aaae5b014673 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:22.832364 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed5c8e3-db48-40cd-870c-aaae5b014673 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:22.833834 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 495/1003] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:02:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:02:22.838790 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d76a5f7-6d7b-457a-b821-ec3b7b82be76 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:22.839085 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d76a5f7-6d7b-457a-b821-ec3b7b82be76 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:22.839392 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d76a5f7-6d7b-457a-b821-ec3b7b82be76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:22.857716 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-1d76a5f7-6d7b-457a-b821-ec3b7b82be76 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:02:22.858896 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-1d76a5f7-6d7b-457a-b821-ec3b7b82be76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:02:22.935095 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 509/1004] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:02:22 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 12:02:23.204069 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed5c8e3-db48-40cd-870c-aaae5b014673 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:23.309938 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed5c8e3-db48-40cd-870c-aaae5b014673 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:23.345522 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc6a94-feba-4120-be85-4ef67a005a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:23.346162 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc6a94-feba-4120-be85-4ef67a005a3e None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:23.346440 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc6a94-feba-4120-be85-4ef67a005a3e None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:23.346708 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc6a94-feba-4120-be85-4ef67a005a3e None admin] PATH_INFO: `/v3/services/image` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:23.347315 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dc6a94-feba-4120-be85-4ef67a005a3e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:23.348683 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dc6a94-feba-4120-be85-4ef67a005a3e None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:23.358347 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-a1dc6a94-feba-4120-be85-4ef67a005a3e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 12:02:23.358347 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:23.358347 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 12:02:23.358347 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 12:02:23.358347 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 12:02:23.358347 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 12:02:23.358347 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 12:02:23.358347 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 12:02:23.358347 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 12:02:23.358347 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:02:23.358347 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 12:02:23.358347 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:02:23.358347 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:23.358347 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:23.358347 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:23.358347 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:23.358347 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:23.359379 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:23.359379 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:23.359379 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:23.359379 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:23.359379 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:23.359379 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:23.359379 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 12:02:23.359379 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 12:02:23.359379 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 12:02:23.359379 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 12:02:23.359379 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:23.359379 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:23.359379 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:23.359379 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:23.359379 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:23.359379 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:23.360233 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:23.360233 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:23.360233 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:23.360233 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:23.360233 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:23.360233 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:23.360233 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:23.360233 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:23.360233 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:23.360233 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:23.360233 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 12:02:23.360233 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 12:02:23.360233 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 12:02:23.360233 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:02:23.367688 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 496/1005] 10.222.0.61 () {56 vars in 1105 bytes} [Fri Aug 26 12:02:23 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:02:23.367835 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d76a5f7-6d7b-457a-b821-ec3b7b82be76 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:23.434968 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:23.476040 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d76a5f7-6d7b-457a-b821-ec3b7b82be76 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:23.513016 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb16966-1a2e-4ba2-93cf-4e7c4172568b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:23.513624 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb16966-1a2e-4ba2-93cf-4e7c4172568b None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:23.513902 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb16966-1a2e-4ba2-93cf-4e7c4172568b None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:23.514172 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb16966-1a2e-4ba2-93cf-4e7c4172568b None admin] PATH_INFO: `/v3/services` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:23.514901 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb16966-1a2e-4ba2-93cf-4e7c4172568b None admin] RBAC: Adding query filter params (name=image) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:23.515232 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb16966-1a2e-4ba2-93cf-4e7c4172568b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:23.516593 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb16966-1a2e-4ba2-93cf-4e7c4172568b None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:23.521521 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 510/1006] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:02:23 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:23.527018 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc6a94-feba-4120-be85-4ef67a005a3e None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:23.548088 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:23.601056 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a0e8dc-4bea-4354-b08c-5041af17735c None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:23.601380 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a0e8dc-4bea-4354-b08c-5041af17735c None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:23.601609 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a0e8dc-4bea-4354-b08c-5041af17735c None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:23.603188 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 511/1007] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:02:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 12:02:23.608145 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d18c777-4b91-41ad-b95d-73772eb0aad4 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:23.608425 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d18c777-4b91-41ad-b95d-73772eb0aad4 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:23.608698 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d18c777-4b91-41ad-b95d-73772eb0aad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:23.626573 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-4d18c777-4b91-41ad-b95d-73772eb0aad4 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:02:23.627773 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-4d18c777-4b91-41ad-b95d-73772eb0aad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:02:23.638583 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc6a94-feba-4120-be85-4ef67a005a3e None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:23.673677 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcf0e50-81f0-4c11-88f9-ba237294e23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:23.674263 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf0e50-81f0-4c11-88f9-ba237294e23a None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:23.674543 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf0e50-81f0-4c11-88f9-ba237294e23a None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:23.674813 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf0e50-81f0-4c11-88f9-ba237294e23a None admin] PATH_INFO: `/v3/services` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:23.675474 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcf0e50-81f0-4c11-88f9-ba237294e23a None admin] RBAC: Adding query filter params (type=image) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:23.675773 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcf0e50-81f0-4c11-88f9-ba237294e23a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:23.677140 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcf0e50-81f0-4c11-88f9-ba237294e23a None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:23.683230 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 497/1008] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:02:23 2022] GET /identity/v3/services?type=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:23.710827 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 512/1009] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:02:23 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 12:02:23.986796 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcf0e50-81f0-4c11-88f9-ba237294e23a None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:24.070358 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:24.089295 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcf0e50-81f0-4c11-88f9-ba237294e23a None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:24.125477 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784cc014-4bad-44d0-ac02-d25eb9bc2612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:24.126108 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784cc014-4bad-44d0-ac02-d25eb9bc2612 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:24.126389 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784cc014-4bad-44d0-ac02-d25eb9bc2612 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:24.126659 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784cc014-4bad-44d0-ac02-d25eb9bc2612 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:24.128673 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784cc014-4bad-44d0-ac02-d25eb9bc2612 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:24.128673 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784cc014-4bad-44d0-ac02-d25eb9bc2612 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:24.137836 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-784cc014-4bad-44d0-ac02-d25eb9bc2612 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 12:02:24.137836 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:24.137836 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 12:02:24.137836 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 12:02:24.137836 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 12:02:24.137836 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 12:02:24.137836 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 12:02:24.137836 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 12:02:24.137836 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 12:02:24.137836 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:02:24.137836 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 12:02:24.137836 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:02:24.137836 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:24.137836 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:24.137836 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:24.137836 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:24.137836 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:24.138940 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:24.138940 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:24.138940 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:24.138940 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:24.138940 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:24.138940 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:24.138940 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 12:02:24.138940 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 12:02:24.138940 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 12:02:24.138940 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 12:02:24.138940 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:24.138940 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:24.138940 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:24.138940 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:24.138940 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:24.138940 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:24.139834 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:24.139834 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:24.139834 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:24.139834 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:24.139834 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:24.139834 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:24.139834 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:24.139834 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:24.139834 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:24.139834 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:24.139834 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 12:02:24.139834 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 12:02:24.139834 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 26 12:02:24.139834 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:02:24.143529 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 498/1010] 10.222.0.61 () {56 vars in 1111 bytes} [Fri Aug 26 12:02:23 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:02:24.144914 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d18c777-4b91-41ad-b95d-73772eb0aad4 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:24.190364 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d08e505-2947-472b-bbd7-7bf8afd31b32 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:24.190693 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d08e505-2947-472b-bbd7-7bf8afd31b32 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:24.190910 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d08e505-2947-472b-bbd7-7bf8afd31b32 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:24.192461 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 499/1011] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:02:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 12:02:24.197186 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92b3328-d756-4e5e-a4d5-a5993acd1da7 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:24.197467 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92b3328-d756-4e5e-a4d5-a5993acd1da7 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:24.197738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92b3328-d756-4e5e-a4d5-a5993acd1da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:24.203197 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:24.217149 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-f92b3328-d756-4e5e-a4d5-a5993acd1da7 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:02:24.218875 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-f92b3328-d756-4e5e-a4d5-a5993acd1da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:02:24.251534 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d18c777-4b91-41ad-b95d-73772eb0aad4 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:24.289045 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 500/1012] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:02:24 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 12:02:24.290898 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73acf19-8709-4cba-8351-bf13540c4a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:24.291538 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73acf19-8709-4cba-8351-bf13540c4a72 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:24.291818 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73acf19-8709-4cba-8351-bf13540c4a72 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:24.292088 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73acf19-8709-4cba-8351-bf13540c4a72 None admin] PATH_INFO: `/v3/services` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:24.292718 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73acf19-8709-4cba-8351-bf13540c4a72 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:24.293020 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73acf19-8709-4cba-8351-bf13540c4a72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:24.294460 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73acf19-8709-4cba-8351-bf13540c4a72 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:24.304210 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 513/1013] 10.222.0.61 () {56 vars in 1112 bytes} [Fri Aug 26 12:02:24 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:24.305134 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92b3328-d756-4e5e-a4d5-a5993acd1da7 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:24.384808 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:24.409693 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92b3328-d756-4e5e-a4d5-a5993acd1da7 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:24.444202 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815a730c-c746-45bf-86b4-3bd4b26c94fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:24.444862 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815a730c-c746-45bf-86b4-3bd4b26c94fd None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:24.445140 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815a730c-c746-45bf-86b4-3bd4b26c94fd None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:24.445410 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815a730c-c746-45bf-86b4-3bd4b26c94fd None admin] PATH_INFO: `/v3/services` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:24.446077 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815a730c-c746-45bf-86b4-3bd4b26c94fd None admin] RBAC: Adding query filter params (type=compute) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:24.446386 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815a730c-c746-45bf-86b4-3bd4b26c94fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:24.447776 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815a730c-c746-45bf-86b4-3bd4b26c94fd None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:24.457253 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 501/1014] 10.222.0.61 () {56 vars in 1112 bytes} [Fri Aug 26 12:02:24 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:02:24.458353 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73acf19-8709-4cba-8351-bf13540c4a72 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:24.558043 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73acf19-8709-4cba-8351-bf13540c4a72 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:24.558449 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815a730c-c746-45bf-86b4-3bd4b26c94fd None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:24.578761 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:24.595466 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da1a681-c47d-4776-a4d4-e844687c7424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:24.596117 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1a681-c47d-4776-a4d4-e844687c7424 None admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:24.596349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1a681-c47d-4776-a4d4-e844687c7424 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:24.596618 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da1a681-c47d-4776-a4d4-e844687c7424 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:24.597275 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da1a681-c47d-4776-a4d4-e844687c7424 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:24.598618 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da1a681-c47d-4776-a4d4-e844687c7424 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:24.631238 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 514/1015] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:02:24 2022] POST /identity/v3/endpoints => generated 334 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:02:24.676948 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815a730c-c746-45bf-86b4-3bd4b26c94fd None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:24.723039 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6508c2-dbee-44c5-941e-5a5bfd230c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:24.723749 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6508c2-dbee-44c5-941e-5a5bfd230c75 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:24.724023 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6508c2-dbee-44c5-941e-5a5bfd230c75 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:24.724291 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6508c2-dbee-44c5-941e-5a5bfd230c75 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:24.725409 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6508c2-dbee-44c5-941e-5a5bfd230c75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:24.726917 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6508c2-dbee-44c5-941e-5a5bfd230c75 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:24.728806 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 502/1016] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:02:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:02:24.735673 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da1a681-c47d-4776-a4d4-e844687c7424 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:24.849072 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da1a681-c47d-4776-a4d4-e844687c7424 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:24.885568 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1817b86-4c69-42b9-ac2f-688f78028ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:24.886169 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1817b86-4c69-42b9-ac2f-688f78028ad4 None admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:24.886444 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1817b86-4c69-42b9-ac2f-688f78028ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:24.886712 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1817b86-4c69-42b9-ac2f-688f78028ad4 None admin] PATH_INFO: `/v3/projects` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:24.888147 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1817b86-4c69-42b9-ac2f-688f78028ad4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:24.889664 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1817b86-4c69-42b9-ac2f-688f78028ad4 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:24.921534 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 515/1017] 10.222.0.61 () {60 vars in 1141 bytes} [Fri Aug 26 12:02:24 2022] POST /identity/v3/projects => generated 311 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:02:25.660156 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59812ece-1aa2-4247-a210-6135644dfbc5 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:25.660714 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59812ece-1aa2-4247-a210-6135644dfbc5 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:25.660948 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59812ece-1aa2-4247-a210-6135644dfbc5 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:25.662401 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 503/1018] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:02:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:02:25.667864 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d46dac3-94ce-40ef-a438-860205e4a837 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:25.669289 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d46dac3-94ce-40ef-a438-860205e4a837 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:25.669289 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d46dac3-94ce-40ef-a438-860205e4a837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:25.687615 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-8d46dac3-94ce-40ef-a438-860205e4a837 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:02:25.688789 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-8d46dac3-94ce-40ef-a438-860205e4a837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:02:25.712687 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:25.763656 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 516/1019] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:02:25 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 12:02:26.035403 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59812ece-1aa2-4247-a210-6135644dfbc5 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:26.039389 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:26.144038 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59812ece-1aa2-4247-a210-6135644dfbc5 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:26.177140 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011ddf37-f524-43df-96f8-1d92e87ea25f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:26.177736 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ddf37-f524-43df-96f8-1d92e87ea25f None admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:26.178012 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ddf37-f524-43df-96f8-1d92e87ea25f None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:26.178283 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ddf37-f524-43df-96f8-1d92e87ea25f None admin] PATH_INFO: `/v3/services` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:26.178930 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011ddf37-f524-43df-96f8-1d92e87ea25f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:26.180317 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011ddf37-f524-43df-96f8-1d92e87ea25f None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:26.198365 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 504/1020] 10.222.0.61 () {60 vars in 1141 bytes} [Fri Aug 26 12:02:26 2022] POST /identity/v3/services => generated 242 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:02:26.682923 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c35f64-9981-4b82-b8a4-d34d66a2e8fe None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:26.683661 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c35f64-9981-4b82-b8a4-d34d66a2e8fe None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:26.683661 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c35f64-9981-4b82-b8a4-d34d66a2e8fe None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:26.685577 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 517/1021] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:02:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:02:26.691174 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f8e66-7f60-445a-9837-0b20ec8ef07e None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:26.691650 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f8e66-7f60-445a-9837-0b20ec8ef07e None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:26.691743 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f8e66-7f60-445a-9837-0b20ec8ef07e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:26.711724 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-cd5f8e66-7f60-445a-9837-0b20ec8ef07e None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:02:26.712927 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-cd5f8e66-7f60-445a-9837-0b20ec8ef07e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:02:26.782929 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:26.799791 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 505/1022] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:02:26 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 12:02:26.944486 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c593e-e5ae-4a9d-a9ef-941cb7be8c67 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:26.944780 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c593e-e5ae-4a9d-a9ef-941cb7be8c67 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:26.944947 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917c593e-e5ae-4a9d-a9ef-941cb7be8c67 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:26.946415 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 518/1023] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:02:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:02:26.951826 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca61658-41e0-4fd2-99c0-56d799f1acb7 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:26.952110 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca61658-41e0-4fd2-99c0-56d799f1acb7 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:26.952379 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca61658-41e0-4fd2-99c0-56d799f1acb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:26.970658 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-4ca61658-41e0-4fd2-99c0-56d799f1acb7 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:02:26.971846 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-4ca61658-41e0-4fd2-99c0-56d799f1acb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:02:27.037109 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 506/1024] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:02:26 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 12:02:27.073016 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917c593e-e5ae-4a9d-a9ef-941cb7be8c67 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:27.178259 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917c593e-e5ae-4a9d-a9ef-941cb7be8c67 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:27.219404 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae89f514-a7f2-4eeb-8917-c7eebaf65719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:27.219955 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae89f514-a7f2-4eeb-8917-c7eebaf65719 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:27.220236 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae89f514-a7f2-4eeb-8917-c7eebaf65719 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:27.220509 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae89f514-a7f2-4eeb-8917-c7eebaf65719 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:27.231268 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae89f514-a7f2-4eeb-8917-c7eebaf65719 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:02:27.236099 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-ae89f514-a7f2-4eeb-8917-c7eebaf65719 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 12:02:27.236099 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:27.236099 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:27.236099 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:27.236099 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:27.236099 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:27.236099 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:27.236099 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:27.236099 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:27.236099 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:27.236099 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:27.236099 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:27.236099 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:02:27.236099 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:02:27.236099 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:02:27.236099 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:02:27.237041 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:02:27.237041 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:02:27.237041 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:02:27.237041 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:02:27.237041 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:27.237041 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:27.237041 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:27.237041 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:27.237041 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:27.237041 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:27.237041 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:27.237041 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:27.237041 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:27.237041 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:27.237041 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:27.237041 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:27.237882 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:27.237882 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:27.237882 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:27.237882 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:27.237882 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:02:27.237882 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:02:27.237882 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:02:27.237882 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:02:27.237882 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:02:27.237882 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:02:27.237882 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 12:02:27.237882 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:27.237882 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 519/1025] 10.222.0.61 () {56 vars in 1120 bytes} [Fri Aug 26 12:02:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 12:02:27.243091 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca61658-41e0-4fd2-99c0-56d799f1acb7 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:27.310477 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae89f514-a7f2-4eeb-8917-c7eebaf65719 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:27.332470 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:27.350978 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca61658-41e0-4fd2-99c0-56d799f1acb7 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:27.373597 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:27.390112 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d323018-3b07-4539-8060-7e204e09d101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:27.390715 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d323018-3b07-4539-8060-7e204e09d101 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:27.390997 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d323018-3b07-4539-8060-7e204e09d101 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:27.391296 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d323018-3b07-4539-8060-7e204e09d101 None admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:27.391969 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d323018-3b07-4539-8060-7e204e09d101 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:27.392271 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d323018-3b07-4539-8060-7e204e09d101 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:27.393646 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d323018-3b07-4539-8060-7e204e09d101 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:27.402734 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 507/1026] 10.222.0.61 () {56 vars in 1115 bytes} [Fri Aug 26 12:02:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:27.408245 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d323018-3b07-4539-8060-7e204e09d101 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:27.428739 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae89f514-a7f2-4eeb-8917-c7eebaf65719 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:27.466575 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c209c6e9-8d8e-4c9e-8832-f8adc360f669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:27.467397 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c209c6e9-8d8e-4c9e-8832-f8adc360f669 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:27.467681 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c209c6e9-8d8e-4c9e-8832-f8adc360f669 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:27.468071 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c209c6e9-8d8e-4c9e-8832-f8adc360f669 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:27.469173 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c209c6e9-8d8e-4c9e-8832-f8adc360f669 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:27.470777 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c209c6e9-8d8e-4c9e-8832-f8adc360f669 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:27.475359 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 520/1027] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:02:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:02:27.480080 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c209c6e9-8d8e-4c9e-8832-f8adc360f669 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:27.516807 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d323018-3b07-4539-8060-7e204e09d101 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:27.555020 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354067b-05b5-4673-9f53-6e0dc4b39dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:27.555663 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354067b-05b5-4673-9f53-6e0dc4b39dc4 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:27.555937 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354067b-05b5-4673-9f53-6e0dc4b39dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:27.556204 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354067b-05b5-4673-9f53-6e0dc4b39dc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:27.567546 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3354067b-05b5-4673-9f53-6e0dc4b39dc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:27.568976 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3354067b-05b5-4673-9f53-6e0dc4b39dc4 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:27.574345 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:27.585205 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-3354067b-05b5-4673-9f53-6e0dc4b39dc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:02:27.585205 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:27.585205 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:27.585205 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:27.585205 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:27.585205 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:27.585205 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:27.585205 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:27.585205 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:27.585205 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:27.585205 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:27.585205 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:27.585205 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:02:27.585205 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:02:27.585205 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:02:27.585205 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:02:27.586200 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:27.586200 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:27.586200 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:27.586200 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:27.586200 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:27.586200 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:27.586200 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:27.586200 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:27.586200 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:27.586200 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:27.586200 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:27.586200 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:27.586200 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:27.586200 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:27.586200 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:27.586200 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:27.586200 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:02:27.587103 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:02:27.587103 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:02:27.587103 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:02:27.587103 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 508/1028] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:02:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 12:02:27.592830 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354067b-05b5-4673-9f53-6e0dc4b39dc4 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:27.595044 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c209c6e9-8d8e-4c9e-8832-f8adc360f669 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:27.631382 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65627374-2e70-4e69-94f1-a4687226f013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:27.631990 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65627374-2e70-4e69-94f1-a4687226f013 None admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:27.632272 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65627374-2e70-4e69-94f1-a4687226f013 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:27.632546 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65627374-2e70-4e69-94f1-a4687226f013 None admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:27.633447 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65627374-2e70-4e69-94f1-a4687226f013 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:27.634335 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:27.634871 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65627374-2e70-4e69-94f1-a4687226f013 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:27.659543 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 521/1029] 10.222.0.61 () {60 vars in 1132 bytes} [Fri Aug 26 12:02:27 2022] POST /identity/v3/users => generated 285 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:02:27.728194 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354067b-05b5-4673-9f53-6e0dc4b39dc4 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:27.769611 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3f068b-f8c7-4c18-b740-0f770734b5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:27.770225 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f068b-f8c7-4c18-b740-0f770734b5e0 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:27.770499 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f068b-f8c7-4c18-b740-0f770734b5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:27.770765 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3f068b-f8c7-4c18-b740-0f770734b5e0 None admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:27.771457 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3f068b-f8c7-4c18-b740-0f770734b5e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:27.771743 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3f068b-f8c7-4c18-b740-0f770734b5e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:27.773267 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3f068b-f8c7-4c18-b740-0f770734b5e0 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:27.787283 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 509/1030] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:02:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:02:27.793135 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65627374-2e70-4e69-94f1-a4687226f013 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:27.907863 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65627374-2e70-4e69-94f1-a4687226f013 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:27.943551 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdf3c53-cd06-4159-bc67-547121519b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:27.944186 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf3c53-cd06-4159-bc67-547121519b95 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:27.944461 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf3c53-cd06-4159-bc67-547121519b95 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:27.944728 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf3c53-cd06-4159-bc67-547121519b95 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:27.952834 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdf3c53-cd06-4159-bc67-547121519b95 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:27.954268 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdf3c53-cd06-4159-bc67-547121519b95 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:27.965434 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-ecdf3c53-cd06-4159-bc67-547121519b95 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 12:02:27.965434 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:27.965434 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:27.965434 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:27.965434 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:27.965434 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:27.965434 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:27.965434 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:27.965434 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:27.965434 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:27.965434 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:27.965434 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:27.965434 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 12:02:27.965434 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 12:02:27.965434 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 12:02:27.965434 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 12:02:27.966428 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:27.966428 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:27.966428 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 12:02:27.966428 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:02:27.966428 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 12:02:27.966428 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:02:27.966428 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:27.966428 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:27.966428 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:27.966428 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:27.966428 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:27.966428 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:27.966428 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:27.966428 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:27.966428 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:27.966428 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:27.966428 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:27.966428 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:27.967848 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:27.967848 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:27.967848 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 12:02:27.967848 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 12:02:27.967848 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 12:02:27.967848 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 12:02:27.967848 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 12:02:27.967848 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 12:02:27.967848 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 12:02:27.967848 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:27.968696 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 522/1031] 10.222.0.61 () {56 vars in 1093 bytes} [Fri Aug 26 12:02:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:02:27.973407 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3f068b-f8c7-4c18-b740-0f770734b5e0 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:28.074663 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:28.082354 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3f068b-f8c7-4c18-b740-0f770734b5e0 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:28.116802 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7954257f-6f9c-42c9-a6c9-b69155f26317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:28.117402 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7954257f-6f9c-42c9-a6c9-b69155f26317 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:28.117675 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7954257f-6f9c-42c9-a6c9-b69155f26317 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:28.117937 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7954257f-6f9c-42c9-a6c9-b69155f26317 None admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:28.118656 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7954257f-6f9c-42c9-a6c9-b69155f26317 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:28.118951 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7954257f-6f9c-42c9-a6c9-b69155f26317 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:28.120388 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7954257f-6f9c-42c9-a6c9-b69155f26317 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:28.139586 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 510/1032] 10.222.0.61 () {56 vars in 1133 bytes} [Fri Aug 26 12:02:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:28.145244 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdf3c53-cd06-4159-bc67-547121519b95 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:28.148208 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:28.207393 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626b601f-637d-4119-96e9-8449826d8e5e None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:28.207588 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626b601f-637d-4119-96e9-8449826d8e5e None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:28.207838 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626b601f-637d-4119-96e9-8449826d8e5e None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:28.209360 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 511/1033] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:02:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 12:02:28.214327 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e08217e-1d58-4d11-942a-2a9387727d0f None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:28.214612 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e08217e-1d58-4d11-942a-2a9387727d0f None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:28.214927 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e08217e-1d58-4d11-942a-2a9387727d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:28.236786 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-1e08217e-1d58-4d11-942a-2a9387727d0f None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:02:28.237098 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-1e08217e-1d58-4d11-942a-2a9387727d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:02:28.266824 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdf3c53-cd06-4159-bc67-547121519b95 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:28.303025 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89432244-9e4f-4d99-a459-45cb952f7e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:28.303668 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89432244-9e4f-4d99-a459-45cb952f7e19 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:28.303941 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89432244-9e4f-4d99-a459-45cb952f7e19 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:28.304209 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89432244-9e4f-4d99-a459-45cb952f7e19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:28.310090 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 512/1034] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:02:28 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 12:02:28.375939 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89432244-9e4f-4d99-a459-45cb952f7e19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:28.377386 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89432244-9e4f-4d99-a459-45cb952f7e19 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:28.379163 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:28.479358 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 523/1035] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:02:28 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 12:02:28.486043 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e08217e-1d58-4d11-942a-2a9387727d0f None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:28.569051 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:28.588624 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89432244-9e4f-4d99-a459-45cb952f7e19 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:28.588782 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e08217e-1d58-4d11-942a-2a9387727d0f None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:28.630228 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04385ad-0e54-4552-b8e2-3c56e33d9e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:28.630771 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04385ad-0e54-4552-b8e2-3c56e33d9e05 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:28.631049 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04385ad-0e54-4552-b8e2-3c56e33d9e05 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:28.631352 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04385ad-0e54-4552-b8e2-3c56e33d9e05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:28.644434 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04385ad-0e54-4552-b8e2-3c56e33d9e05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:28.645882 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04385ad-0e54-4552-b8e2-3c56e33d9e05 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:28.661175 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-c04385ad-0e54-4552-b8e2-3c56e33d9e05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:02:28.661175 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:28.661175 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:28.661175 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:28.661175 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:28.661175 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:28.661175 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:28.661175 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:28.661175 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:28.661175 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:28.661175 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:28.661175 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:28.661175 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:02:28.661175 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:02:28.661175 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:02:28.661175 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:02:28.662156 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:28.662156 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:28.662156 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:28.662156 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:28.662156 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:28.662156 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:28.662156 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:28.662156 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:28.662156 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:28.662156 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:28.662156 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:28.662156 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:28.662156 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:28.662156 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:28.662156 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:28.662156 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:28.662156 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:02:28.663060 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:02:28.663060 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:02:28.663060 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:02:28.663060 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 513/1036] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:02:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:02:28.667976 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04385ad-0e54-4552-b8e2-3c56e33d9e05 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:28.698104 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89432244-9e4f-4d99-a459-45cb952f7e19 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:28.700276 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:28.735945 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56344c2-6b77-4953-be2c-4e060d791950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:28.736580 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56344c2-6b77-4953-be2c-4e060d791950 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:28.736865 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56344c2-6b77-4953-be2c-4e060d791950 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:28.737139 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56344c2-6b77-4953-be2c-4e060d791950 None admin] PATH_INFO: `/v3/services/image` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:28.737823 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56344c2-6b77-4953-be2c-4e060d791950 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:28.739338 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56344c2-6b77-4953-be2c-4e060d791950 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:28.748614 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-e56344c2-6b77-4953-be2c-4e060d791950 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 12:02:28.748614 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:28.748614 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 12:02:28.748614 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 12:02:28.748614 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 12:02:28.748614 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 12:02:28.748614 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 12:02:28.748614 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 12:02:28.748614 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 12:02:28.748614 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:28.748614 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 12:02:28.748614 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:28.748614 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:28.748614 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:28.748614 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:28.748614 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:28.748614 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:28.749770 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:28.749770 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:28.749770 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:28.749770 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:28.749770 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:28.749770 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:28.749770 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 12:02:28.749770 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 12:02:28.749770 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 12:02:28.749770 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 12:02:28.749770 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:28.749770 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:28.749770 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:28.749770 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:28.749770 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:28.749770 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:28.750627 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:28.750627 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:28.750627 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:28.750627 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:28.750627 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:28.750627 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:28.750627 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:28.750627 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:28.750627 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:28.750627 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:28.750627 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 12:02:28.750627 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 12:02:28.750627 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 12:02:28.750627 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:28.750627 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 524/1037] 10.222.0.61 () {56 vars in 1105 bytes} [Fri Aug 26 12:02:28 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 12:02:28.756079 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56344c2-6b77-4953-be2c-4e060d791950 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:28.782610 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04385ad-0e54-4552-b8e2-3c56e33d9e05 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:28.819463 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c1f9b-fd0b-4772-a9dc-610f63e006b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:28.820818 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c1f9b-fd0b-4772-a9dc-610f63e006b8 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:28.821328 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c1f9b-fd0b-4772-a9dc-610f63e006b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:28.822073 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c1f9b-fd0b-4772-a9dc-610f63e006b8 None admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:28.823131 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820c1f9b-fd0b-4772-a9dc-610f63e006b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:28.823625 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820c1f9b-fd0b-4772-a9dc-610f63e006b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:28.826244 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820c1f9b-fd0b-4772-a9dc-610f63e006b8 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:28.828852 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:28.842228 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 514/1038] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:02:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:02:28.848708 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c1f9b-fd0b-4772-a9dc-610f63e006b8 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:28.875790 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56344c2-6b77-4953-be2c-4e060d791950 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:28.914349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620fec0a-44a8-4b63-b710-ef9e01622767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:28.914881 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620fec0a-44a8-4b63-b710-ef9e01622767 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:28.915294 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620fec0a-44a8-4b63-b710-ef9e01622767 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:28.915568 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620fec0a-44a8-4b63-b710-ef9e01622767 None admin] PATH_INFO: `/v3/services` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:28.916220 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620fec0a-44a8-4b63-b710-ef9e01622767 None admin] RBAC: Adding query filter params (name=image) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:28.916604 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620fec0a-44a8-4b63-b710-ef9e01622767 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:28.917964 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620fec0a-44a8-4b63-b710-ef9e01622767 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:28.929134 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 525/1039] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:02:28 2022] GET /identity/v3/services?name=image => generated 122 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:02:28.930015 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620fec0a-44a8-4b63-b710-ef9e01622767 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:28.951579 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c1f9b-fd0b-4772-a9dc-610f63e006b8 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:28.961806 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:28.991291 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e08b083-0dda-4414-aa84-a2d839f2167b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:28.992032 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e08b083-0dda-4414-aa84-a2d839f2167b None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:28.992311 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e08b083-0dda-4414-aa84-a2d839f2167b None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:28.992587 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e08b083-0dda-4414-aa84-a2d839f2167b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:29.001662 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e08b083-0dda-4414-aa84-a2d839f2167b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:29.003341 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e08b083-0dda-4414-aa84-a2d839f2167b None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:29.015278 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-5e08b083-0dda-4414-aa84-a2d839f2167b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:02:29.015278 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:29.015278 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:29.015278 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:29.015278 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:29.015278 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:29.015278 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:29.015278 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:29.015278 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:29.015278 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:29.015278 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:29.015278 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:29.015278 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 12:02:29.015278 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 12:02:29.015278 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 12:02:29.016209 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 12:02:29.016209 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:29.016209 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:29.016209 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:29.016209 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:29.016209 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:29.016209 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:29.016209 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:29.016209 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:29.016209 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:29.016209 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:29.016209 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:29.016209 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:29.016209 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:29.016209 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:29.016209 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:29.016209 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:29.017119 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 12:02:29.017119 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 12:02:29.017119 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 12:02:29.017119 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 12:02:29.017119 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 12:02:29.017119 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 12:02:29.017119 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:02:29.017119 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:02:29.017119 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 515/1040] 10.222.0.61 () {56 vars in 1111 bytes} [Fri Aug 26 12:02:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 12:02:29.022300 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e08b083-0dda-4414-aa84-a2d839f2167b None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:29.038790 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620fec0a-44a8-4b63-b710-ef9e01622767 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:29.073738 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4411410c-564f-4080-9965-c2f6ad695ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:29.074494 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411410c-564f-4080-9965-c2f6ad695ed1 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:29.074861 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411410c-564f-4080-9965-c2f6ad695ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:29.075224 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4411410c-564f-4080-9965-c2f6ad695ed1 None admin] PATH_INFO: `/v3/services` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:29.075873 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4411410c-564f-4080-9965-c2f6ad695ed1 None admin] RBAC: Adding query filter params (type=image) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:29.076177 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4411410c-564f-4080-9965-c2f6ad695ed1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:29.077544 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4411410c-564f-4080-9965-c2f6ad695ed1 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:29.081512 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:29.083320 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 526/1041] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:02:28 2022] GET /identity/v3/services?type=image => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:29.089154 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4411410c-564f-4080-9965-c2f6ad695ed1 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:29.131905 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e08b083-0dda-4414-aa84-a2d839f2167b None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:29.169842 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9031e4da-5f4f-4b16-88a8-ef59f6269c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:29.170729 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9031e4da-5f4f-4b16-88a8-ef59f6269c6f None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:29.171194 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9031e4da-5f4f-4b16-88a8-ef59f6269c6f None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:29.171672 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9031e4da-5f4f-4b16-88a8-ef59f6269c6f None admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:29.172574 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9031e4da-5f4f-4b16-88a8-ef59f6269c6f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:29.173200 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9031e4da-5f4f-4b16-88a8-ef59f6269c6f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:29.174655 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9031e4da-5f4f-4b16-88a8-ef59f6269c6f None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:29.189028 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 516/1042] 10.222.0.61 () {56 vars in 1148 bytes} [Fri Aug 26 12:02:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:02:29.194568 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9031e4da-5f4f-4b16-88a8-ef59f6269c6f None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:29.198161 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:29.205648 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4411410c-564f-4080-9965-c2f6ad695ed1 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:29.241559 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00433323-ae73-41d1-b99a-b0be1d75cf56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:29.242175 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00433323-ae73-41d1-b99a-b0be1d75cf56 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:29.242447 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00433323-ae73-41d1-b99a-b0be1d75cf56 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:29.242720 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00433323-ae73-41d1-b99a-b0be1d75cf56 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:29.243491 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00433323-ae73-41d1-b99a-b0be1d75cf56 None admin] RBAC: Adding query filter params (interface=public, service_id=0c9e59ce04f147cbbb771155bc725838, region_id=RegionOne) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:29.243797 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00433323-ae73-41d1-b99a-b0be1d75cf56 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:29.245154 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00433323-ae73-41d1-b99a-b0be1d75cf56 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:29.250893 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 527/1043] 10.222.0.61 () {56 vars in 1251 bytes} [Fri Aug 26 12:02:29 2022] GET /identity/v3/endpoints?service_id=0c9e59ce04f147cbbb771155bc725838&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:02:29.256380 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00433323-ae73-41d1-b99a-b0be1d75cf56 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:29.315683 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9031e4da-5f4f-4b16-88a8-ef59f6269c6f None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:29.330279 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:29.352122 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e20960-b81d-4885-8c09-21d5a5227f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:29.352751 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e20960-b81d-4885-8c09-21d5a5227f87 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:29.353038 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e20960-b81d-4885-8c09-21d5a5227f87 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:29.353313 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e20960-b81d-4885-8c09-21d5a5227f87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:29.354740 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e20960-b81d-4885-8c09-21d5a5227f87 None admin] RBAC: Adding query filter params (role.id=ebae7fe0ca914cd1a0106219ed37c042, scope.project.id=ecd68423a83a427681d44dbf266fce6e, user.id=7aacd7d6d17e4f10ba5c9107a12beafa) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:29.355062 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e20960-b81d-4885-8c09-21d5a5227f87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:29.357302 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e20960-b81d-4885-8c09-21d5a5227f87 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:29.363388 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 517/1044] 10.222.0.61 () {56 vars in 1374 bytes} [Fri Aug 26 12:02:29 2022] GET /identity/v3/role_assignments?user.id=7aacd7d6d17e4f10ba5c9107a12beafa&scope.project.id=ecd68423a83a427681d44dbf266fce6e&role.id=ebae7fe0ca914cd1a0106219ed37c042 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:02:29.366170 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00433323-ae73-41d1-b99a-b0be1d75cf56 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:29.406837 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f16272-b1e7-438e-b9d5-342af3d0a469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:29.407495 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f16272-b1e7-438e-b9d5-342af3d0a469 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:29.407777 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f16272-b1e7-438e-b9d5-342af3d0a469 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:29.408054 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f16272-b1e7-438e-b9d5-342af3d0a469 None admin] PATH_INFO: `/v3/services` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:29.408752 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f16272-b1e7-438e-b9d5-342af3d0a469 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:29.410190 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f16272-b1e7-438e-b9d5-342af3d0a469 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:29.417975 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 528/1045] 10.222.0.61 () {56 vars in 1087 bytes} [Fri Aug 26 12:02:29 2022] GET /identity/v3/services => generated 1957 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:02:29.688602 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61903987-3ad1-4921-b336-32b753f508be None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:29.688937 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61903987-3ad1-4921-b336-32b753f508be None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:29.689146 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61903987-3ad1-4921-b336-32b753f508be None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:29.690622 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 518/1046] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:02:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:02:29.695625 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba59a8-7ae4-4e56-9609-88649d49c5cc None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:29.695913 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba59a8-7ae4-4e56-9609-88649d49c5cc None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:29.696186 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba59a8-7ae4-4e56-9609-88649d49c5cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:29.714244 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-32ba59a8-7ae4-4e56-9609-88649d49c5cc None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:02:29.716009 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-32ba59a8-7ae4-4e56-9609-88649d49c5cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:02:29.734246 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:29.796238 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 529/1047] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:02:29 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 12:02:30.075864 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61903987-3ad1-4921-b336-32b753f508be None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:30.180380 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61903987-3ad1-4921-b336-32b753f508be None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:30.213999 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1f175b-f10d-4483-a36e-105ba333d50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:30.214613 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f175b-f10d-4483-a36e-105ba333d50a None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:30.214890 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f175b-f10d-4483-a36e-105ba333d50a None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:30.215900 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f175b-f10d-4483-a36e-105ba333d50a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:30.225854 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1f175b-f10d-4483-a36e-105ba333d50a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:02:30.230441 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-4c1f175b-f10d-4483-a36e-105ba333d50a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 12:02:30.230441 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:30.230441 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:30.230441 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:30.230441 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:30.230441 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:30.230441 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:30.230441 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:30.230441 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:30.230441 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:30.230441 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:30.230441 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:30.230441 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:02:30.230441 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:02:30.230441 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:02:30.230441 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:02:30.231756 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:02:30.231756 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:02:30.231756 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:02:30.231756 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:02:30.231756 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:30.231756 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:30.231756 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:30.231756 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:30.231756 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:30.231756 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:30.231756 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:30.231756 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:30.231756 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:30.231756 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:30.231756 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:30.231756 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:30.232612 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:30.232612 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:30.232612 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:30.232612 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:30.232612 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:02:30.232612 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:02:30.232612 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:02:30.232612 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:02:30.232612 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:02:30.232612 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:02:30.232612 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 12:02:30.232612 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:02:30.235529 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 519/1048] 10.222.0.61 () {56 vars in 1096 bytes} [Fri Aug 26 12:02:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:02:30.237607 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ba59a8-7ae4-4e56-9609-88649d49c5cc None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:30.337647 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ba59a8-7ae4-4e56-9609-88649d49c5cc None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:30.371628 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9880de2c-d02f-4025-9fd3-c64ed547e120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:30.372442 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9880de2c-d02f-4025-9fd3-c64ed547e120 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:30.372721 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9880de2c-d02f-4025-9fd3-c64ed547e120 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:30.372991 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9880de2c-d02f-4025-9fd3-c64ed547e120 None admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:30.373653 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9880de2c-d02f-4025-9fd3-c64ed547e120 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:30.373951 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9880de2c-d02f-4025-9fd3-c64ed547e120 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:30.375326 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9880de2c-d02f-4025-9fd3-c64ed547e120 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:30.384043 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 530/1049] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:02:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:30.390447 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1f175b-f10d-4483-a36e-105ba333d50a None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:30.431931 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:30.495535 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1f175b-f10d-4483-a36e-105ba333d50a None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:30.532593 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16ad5c8-bf3b-4ec6-9d82-d5d15067144f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:30.533216 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ad5c8-bf3b-4ec6-9d82-d5d15067144f None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:30.533490 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ad5c8-bf3b-4ec6-9d82-d5d15067144f None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:30.533757 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ad5c8-bf3b-4ec6-9d82-d5d15067144f None admin] PATH_INFO: `/v3/users/07ce16fc38a9414889c6c1b290622b79` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:30.547628 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16ad5c8-bf3b-4ec6-9d82-d5d15067144f None admin] RBAC: Authorizing `identity:get_user(user_id=07ce16fc38a9414889c6c1b290622b79)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:30.549043 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16ad5c8-bf3b-4ec6-9d82-d5d15067144f None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:30.550787 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 520/1050] 10.222.0.61 () {56 vars in 1177 bytes} [Fri Aug 26 12:02:30 2022] GET /identity/v3/users/07ce16fc38a9414889c6c1b290622b79 => generated 285 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:30.556572 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9880de2c-d02f-4025-9fd3-c64ed547e120 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:30.573691 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:30.658602 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9880de2c-d02f-4025-9fd3-c64ed547e120 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:30.692101 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ec86a5-fc7f-4188-ad64-71f45e8177b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:30.692700 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec86a5-fc7f-4188-ad64-71f45e8177b4 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:30.692973 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec86a5-fc7f-4188-ad64-71f45e8177b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:30.693239 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec86a5-fc7f-4188-ad64-71f45e8177b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:30.768903 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ec86a5-fc7f-4188-ad64-71f45e8177b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:30.770349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ec86a5-fc7f-4188-ad64-71f45e8177b4 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:30.784988 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:30.870310 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 531/1051] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:02:30 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 26 12:02:30.876582 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16ad5c8-bf3b-4ec6-9d82-d5d15067144f None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:30.970844 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:30.978261 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16ad5c8-bf3b-4ec6-9d82-d5d15067144f None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:31.012666 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5470ad-d3ab-4bbe-be68-9147a65603c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:31.013295 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5470ad-d3ab-4bbe-be68-9147a65603c9 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:31.013623 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5470ad-d3ab-4bbe-be68-9147a65603c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:31.013851 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5470ad-d3ab-4bbe-be68-9147a65603c9 None admin] PATH_INFO: `/v3/projects/5367531f36164255a59bd78a9d4f1b0c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:31.014957 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5470ad-d3ab-4bbe-be68-9147a65603c9 None admin] RBAC: Authorizing `identity:get_project(project_id=5367531f36164255a59bd78a9d4f1b0c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:31.016410 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5470ad-d3ab-4bbe-be68-9147a65603c9 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:31.018209 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 521/1052] 10.222.0.61 () {56 vars in 1186 bytes} [Fri Aug 26 12:02:30 2022] GET /identity/v3/projects/5367531f36164255a59bd78a9d4f1b0c => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:31.024109 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ec86a5-fc7f-4188-ad64-71f45e8177b4 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:31.122920 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ec86a5-fc7f-4188-ad64-71f45e8177b4 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:31.138221 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:31.159600 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3f48e4-ff17-49e6-b00e-96180a8250be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:31.160197 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f48e4-ff17-49e6-b00e-96180a8250be None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:31.160472 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f48e4-ff17-49e6-b00e-96180a8250be None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:31.160738 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f48e4-ff17-49e6-b00e-96180a8250be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:31.161448 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3f48e4-ff17-49e6-b00e-96180a8250be None admin] RBAC: Adding query filter params (role.id=70ec1ba484714c8ca3adb8fe5845107c, scope.project.id=5367531f36164255a59bd78a9d4f1b0c, user.id=07ce16fc38a9414889c6c1b290622b79) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:31.161748 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3f48e4-ff17-49e6-b00e-96180a8250be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:31.163151 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3f48e4-ff17-49e6-b00e-96180a8250be None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:31.168777 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 532/1053] 10.222.0.61 () {56 vars in 1374 bytes} [Fri Aug 26 12:02:31 2022] GET /identity/v3/role_assignments?user.id=07ce16fc38a9414889c6c1b290622b79&scope.project.id=5367531f36164255a59bd78a9d4f1b0c&role.id=70ec1ba484714c8ca3adb8fe5845107c => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:31.292641 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f792804-ffc0-4233-8f8c-3d45bc3f7ebd None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:31.293140 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f792804-ffc0-4233-8f8c-3d45bc3f7ebd None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:31.293140 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f792804-ffc0-4233-8f8c-3d45bc3f7ebd None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:31.294626 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 522/1054] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:02:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:02:31.299958 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bce110-ab5d-469d-aab3-4f0b94f4e8cd None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:31.300435 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bce110-ab5d-469d-aab3-4f0b94f4e8cd None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:31.300519 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bce110-ab5d-469d-aab3-4f0b94f4e8cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:31.320745 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-73bce110-ab5d-469d-aab3-4f0b94f4e8cd None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:02:31.321952 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-73bce110-ab5d-469d-aab3-4f0b94f4e8cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:02:31.350655 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e46a13-5d73-4e91-83eb-4c0f7005a103 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:31.350981 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e46a13-5d73-4e91-83eb-4c0f7005a103 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:31.351220 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e46a13-5d73-4e91-83eb-4c0f7005a103 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:31.352569 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 523/1055] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:02:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 12:02:31.357747 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6aae87-0df5-4bb4-98b7-f76b9f639c9d None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:31.358033 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6aae87-0df5-4bb4-98b7-f76b9f639c9d None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:31.358295 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6aae87-0df5-4bb4-98b7-f76b9f639c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:31.376190 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-7d6aae87-0df5-4bb4-98b7-f76b9f639c9d None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:02:31.377363 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-7d6aae87-0df5-4bb4-98b7-f76b9f639c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:02:31.394946 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 533/1056] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:02:31 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 12:02:31.445534 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 524/1057] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:02:31 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 12:02:31.665540 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bce110-ab5d-469d-aab3-4f0b94f4e8cd None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:31.712419 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6aae87-0df5-4bb4-98b7-f76b9f639c9d None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:31.777232 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bce110-ab5d-469d-aab3-4f0b94f4e8cd None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:31.788326 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:31.816414 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703afb32-38fa-4ea6-a4ce-bf43e93bcf16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:31.817336 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703afb32-38fa-4ea6-a4ce-bf43e93bcf16 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:31.817798 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703afb32-38fa-4ea6-a4ce-bf43e93bcf16 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:31.818170 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703afb32-38fa-4ea6-a4ce-bf43e93bcf16 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:31.819653 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:31.827462 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6aae87-0df5-4bb4-98b7-f76b9f639c9d None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:31.829096 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703afb32-38fa-4ea6-a4ce-bf43e93bcf16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:02:31.833688 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-703afb32-38fa-4ea6-a4ce-bf43e93bcf16 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 12:02:31.833688 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:31.833688 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:31.833688 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:31.833688 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:31.833688 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:31.833688 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:31.833688 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:31.833688 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:31.833688 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:31.833688 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:31.833688 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:31.833688 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:02:31.833688 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:02:31.833688 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:02:31.833688 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:02:31.834626 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:02:31.834626 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:02:31.834626 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:02:31.834626 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:02:31.834626 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:31.834626 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:31.834626 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:31.834626 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:31.834626 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:31.834626 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:31.834626 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:31.834626 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:31.834626 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:31.834626 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:31.834626 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:31.834626 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:31.836161 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:31.836161 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:31.836161 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:31.836161 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:31.836161 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:02:31.836161 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:02:31.836161 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:02:31.836161 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:02:31.836161 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:02:31.836161 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:02:31.836161 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 12:02:31.836161 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:31.836161 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 534/1058] 10.222.0.61 () {56 vars in 1120 bytes} [Fri Aug 26 12:02:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:02:31.840951 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703afb32-38fa-4ea6-a4ce-bf43e93bcf16 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:31.866050 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8e632b-4dac-4f0e-ad34-36cd86dfe7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:31.866885 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e632b-4dac-4f0e-ad34-36cd86dfe7f9 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:31.867174 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e632b-4dac-4f0e-ad34-36cd86dfe7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:31.867484 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e632b-4dac-4f0e-ad34-36cd86dfe7f9 None admin] PATH_INFO: `/v3/services/image` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:31.868063 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8e632b-4dac-4f0e-ad34-36cd86dfe7f9 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:31.869420 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8e632b-4dac-4f0e-ad34-36cd86dfe7f9 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:31.878532 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-ae8e632b-4dac-4f0e-ad34-36cd86dfe7f9 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 12:02:31.878532 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:31.878532 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 12:02:31.878532 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 12:02:31.878532 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 12:02:31.878532 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 12:02:31.878532 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 12:02:31.878532 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 12:02:31.878532 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 12:02:31.878532 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:02:31.878532 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 12:02:31.878532 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:02:31.878532 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:31.878532 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:31.878532 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:31.878532 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:31.878532 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:31.879574 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:31.879574 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:31.879574 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:31.879574 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:31.879574 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:31.879574 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:31.879574 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 12:02:31.879574 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 12:02:31.879574 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 12:02:31.879574 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 12:02:31.879574 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:31.879574 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:31.879574 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:31.879574 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:31.879574 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:31.879574 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:31.880443 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:31.880443 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:31.880443 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:31.880443 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:31.880443 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:31.880443 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:31.880443 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:31.880443 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:31.880443 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:31.880443 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:31.880443 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 12:02:31.880443 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 12:02:31.880443 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 26 12:02:31.880443 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:02:31.880443 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 525/1059] 10.222.0.61 () {56 vars in 1105 bytes} [Fri Aug 26 12:02:31 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 12:02:31.885764 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8e632b-4dac-4f0e-ad34-36cd86dfe7f9 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:31.952896 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703afb32-38fa-4ea6-a4ce-bf43e93bcf16 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:31.991458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407cc500-b58a-474d-b1a3-a81ad099775b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:31.992061 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407cc500-b58a-474d-b1a3-a81ad099775b None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:31.992333 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407cc500-b58a-474d-b1a3-a81ad099775b None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:31.992599 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407cc500-b58a-474d-b1a3-a81ad099775b None admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:31.993261 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407cc500-b58a-474d-b1a3-a81ad099775b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:31.993562 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407cc500-b58a-474d-b1a3-a81ad099775b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:31.994914 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407cc500-b58a-474d-b1a3-a81ad099775b None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:31.996071 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8e632b-4dac-4f0e-ad34-36cd86dfe7f9 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:32.004326 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 535/1060] 10.222.0.61 () {56 vars in 1115 bytes} [Fri Aug 26 12:02:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:32.010116 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407cc500-b58a-474d-b1a3-a81ad099775b None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:32.035764 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367d4c66-fc4a-4312-98a5-22003ae81342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:32.036659 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367d4c66-fc4a-4312-98a5-22003ae81342 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:32.036938 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367d4c66-fc4a-4312-98a5-22003ae81342 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:32.037206 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367d4c66-fc4a-4312-98a5-22003ae81342 None admin] PATH_INFO: `/v3/services` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:32.037819 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367d4c66-fc4a-4312-98a5-22003ae81342 None admin] RBAC: Adding query filter params (name=image) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:32.038115 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367d4c66-fc4a-4312-98a5-22003ae81342 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:32.039510 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367d4c66-fc4a-4312-98a5-22003ae81342 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:32.044339 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 526/1061] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:02:31 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:02:32.045870 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:32.050133 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367d4c66-fc4a-4312-98a5-22003ae81342 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:32.080701 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:32.118541 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407cc500-b58a-474d-b1a3-a81ad099775b None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:32.156158 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9b5e46-ed1c-4930-9202-7d63d8cf6379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:32.156789 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9b5e46-ed1c-4930-9202-7d63d8cf6379 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:32.157064 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9b5e46-ed1c-4930-9202-7d63d8cf6379 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:32.157331 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9b5e46-ed1c-4930-9202-7d63d8cf6379 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:32.168820 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367d4c66-fc4a-4312-98a5-22003ae81342 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:32.170678 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9b5e46-ed1c-4930-9202-7d63d8cf6379 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:32.172944 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9b5e46-ed1c-4930-9202-7d63d8cf6379 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:32.187548 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-ab9b5e46-ed1c-4930-9202-7d63d8cf6379 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:02:32.187548 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:32.187548 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:32.187548 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:32.187548 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:32.187548 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:32.187548 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:32.187548 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:32.187548 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:32.187548 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:32.187548 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:32.187548 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:32.187548 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:02:32.187548 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:02:32.187548 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:02:32.187548 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:02:32.188559 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:32.188559 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:32.188559 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:32.188559 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:32.188559 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:32.188559 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:32.188559 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:32.188559 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:32.188559 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:32.188559 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:32.188559 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:32.188559 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:32.188559 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:32.188559 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:32.188559 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:32.188559 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:32.188559 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:02:32.189469 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:02:32.189469 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:02:32.189469 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:32.191368 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 536/1062] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:02:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 12:02:32.194190 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9b5e46-ed1c-4930-9202-7d63d8cf6379 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:32.205978 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea908c4-e9ee-4652-b059-65050f85845e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:32.206520 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea908c4-e9ee-4652-b059-65050f85845e None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:32.206799 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea908c4-e9ee-4652-b059-65050f85845e None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:32.207073 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea908c4-e9ee-4652-b059-65050f85845e None admin] PATH_INFO: `/v3/services` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:32.207736 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea908c4-e9ee-4652-b059-65050f85845e None admin] RBAC: Adding query filter params (type=image) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:32.208044 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea908c4-e9ee-4652-b059-65050f85845e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:32.209410 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea908c4-e9ee-4652-b059-65050f85845e None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:32.214361 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 527/1063] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:02:32 2022] GET /identity/v3/services?type=image => generated 350 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:32.220218 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea908c4-e9ee-4652-b059-65050f85845e None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:32.285184 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:32.305446 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9b5e46-ed1c-4930-9202-7d63d8cf6379 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:32.322404 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:32.337565 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea908c4-e9ee-4652-b059-65050f85845e None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:32.342582 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5889ebf-42b9-43b8-9c68-2e2caed3990c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:32.343189 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5889ebf-42b9-43b8-9c68-2e2caed3990c None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:32.343511 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5889ebf-42b9-43b8-9c68-2e2caed3990c None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:32.343759 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5889ebf-42b9-43b8-9c68-2e2caed3990c None admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:32.344763 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5889ebf-42b9-43b8-9c68-2e2caed3990c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:32.344763 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5889ebf-42b9-43b8-9c68-2e2caed3990c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:32.346104 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5889ebf-42b9-43b8-9c68-2e2caed3990c None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:32.359004 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 537/1064] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:02:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:02:32.364316 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5889ebf-42b9-43b8-9c68-2e2caed3990c None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:32.375030 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d178ea9-4b61-4643-889f-598a773527ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:32.376014 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d178ea9-4b61-4643-889f-598a773527ca None admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:32.376294 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d178ea9-4b61-4643-889f-598a773527ca None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:32.376564 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d178ea9-4b61-4643-889f-598a773527ca None admin] PATH_INFO: `/v3/endpoints` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:32.377568 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d178ea9-4b61-4643-889f-598a773527ca None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:32.378886 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d178ea9-4b61-4643-889f-598a773527ca None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:32.399532 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 528/1065] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:02:32 2022] POST /identity/v3/endpoints => generated 327 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 12:02:32.487775 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5889ebf-42b9-43b8-9c68-2e2caed3990c None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:32.532782 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19c2aa3-5813-4ede-b16c-4ced8fe8126b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:32.533425 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19c2aa3-5813-4ede-b16c-4ced8fe8126b None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:32.533706 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19c2aa3-5813-4ede-b16c-4ced8fe8126b None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:32.533972 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19c2aa3-5813-4ede-b16c-4ced8fe8126b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:32.542847 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19c2aa3-5813-4ede-b16c-4ced8fe8126b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:32.544783 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19c2aa3-5813-4ede-b16c-4ced8fe8126b None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:32.547905 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:32.560469 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-b19c2aa3-5813-4ede-b16c-4ced8fe8126b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 12:02:32.560469 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:32.560469 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:32.560469 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:32.560469 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:32.560469 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:32.560469 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:32.560469 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:32.560469 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:32.560469 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:32.560469 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:32.560469 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:32.560469 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 12:02:32.560469 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 12:02:32.560469 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 12:02:32.560469 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 12:02:32.561624 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:32.561624 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:32.561624 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 12:02:32.561624 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:02:32.561624 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 12:02:32.561624 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:02:32.561624 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:32.561624 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:32.561624 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:32.561624 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:32.561624 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:32.561624 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:32.561624 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:32.561624 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:32.561624 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:32.561624 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:32.561624 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:32.561624 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:32.562798 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:32.562798 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:32.562798 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 12:02:32.562798 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 12:02:32.562798 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 12:02:32.562798 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 12:02:32.562798 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 12:02:32.562798 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 12:02:32.562798 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 12:02:32.562798 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:32.564485 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 538/1066] 10.222.0.61 () {56 vars in 1093 bytes} [Fri Aug 26 12:02:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 12:02:32.570572 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d178ea9-4b61-4643-889f-598a773527ca None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:32.675272 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d178ea9-4b61-4643-889f-598a773527ca None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:32.708761 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0d0e46-121a-44f7-98d9-2fa5ae463c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:32.709362 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0d0e46-121a-44f7-98d9-2fa5ae463c63 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:32.709637 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0d0e46-121a-44f7-98d9-2fa5ae463c63 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:32.709905 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0d0e46-121a-44f7-98d9-2fa5ae463c63 None admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:32.710617 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0d0e46-121a-44f7-98d9-2fa5ae463c63 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:32.710916 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0d0e46-121a-44f7-98d9-2fa5ae463c63 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:32.712360 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0d0e46-121a-44f7-98d9-2fa5ae463c63 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:32.726713 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:32.736526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 529/1067] 10.222.0.61 () {56 vars in 1133 bytes} [Fri Aug 26 12:02:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:32.737925 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19c2aa3-5813-4ede-b16c-4ced8fe8126b None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:32.852218 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19c2aa3-5813-4ede-b16c-4ced8fe8126b None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:32.889148 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262d084a-e9b4-43a9-ae29-b0c219ad364f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:32.889779 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d084a-e9b4-43a9-ae29-b0c219ad364f None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:32.890064 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d084a-e9b4-43a9-ae29-b0c219ad364f None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:32.890336 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d084a-e9b4-43a9-ae29-b0c219ad364f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:32.901382 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262d084a-e9b4-43a9-ae29-b0c219ad364f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:32.903345 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262d084a-e9b4-43a9-ae29-b0c219ad364f None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:32.917364 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-262d084a-e9b4-43a9-ae29-b0c219ad364f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:02:32.917364 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:32.917364 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:32.917364 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:32.917364 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:32.917364 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:32.917364 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:32.917364 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:32.917364 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:32.917364 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:32.917364 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:32.917364 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:32.917364 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:02:32.917364 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:02:32.917364 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:02:32.917364 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:02:32.923402 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:32.923402 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:32.923402 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:32.923402 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:32.923402 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:32.923402 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:32.923402 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:32.923402 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:32.923402 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:32.923402 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:32.923402 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:32.923402 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:32.923402 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:32.923402 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:32.923402 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:32.923402 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:32.923402 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:02:32.924381 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:02:32.924381 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:02:32.924381 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:32.924381 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 539/1068] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:02:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 12:02:32.926611 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0d0e46-121a-44f7-98d9-2fa5ae463c63 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:33.028836 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0d0e46-121a-44f7-98d9-2fa5ae463c63 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:33.062852 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9509820-ae70-405c-868a-b2b5659bf491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:33.063483 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9509820-ae70-405c-868a-b2b5659bf491 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:33.063760 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9509820-ae70-405c-868a-b2b5659bf491 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:33.064026 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9509820-ae70-405c-868a-b2b5659bf491 None admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:33.064656 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9509820-ae70-405c-868a-b2b5659bf491 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:33.064951 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9509820-ae70-405c-868a-b2b5659bf491 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:33.066302 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9509820-ae70-405c-868a-b2b5659bf491 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:33.078817 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 530/1069] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:02:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:33.083097 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9509820-ae70-405c-868a-b2b5659bf491 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:33.083981 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf11a9-b1b5-4127-a315-719750c921da None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:33.084261 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf11a9-b1b5-4127-a315-719750c921da None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:33.084527 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf11a9-b1b5-4127-a315-719750c921da None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:33.085846 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 540/1070] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:02:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 12:02:33.090854 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87beb579-a301-4228-a788-e05ca86ddf42 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:33.091126 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87beb579-a301-4228-a788-e05ca86ddf42 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:33.091424 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87beb579-a301-4228-a788-e05ca86ddf42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:33.110945 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-87beb579-a301-4228-a788-e05ca86ddf42 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:02:33.112163 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-87beb579-a301-4228-a788-e05ca86ddf42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:02:33.145295 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:33.160733 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:33.185200 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 541/1071] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:02:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 12:02:33.190240 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9509820-ae70-405c-868a-b2b5659bf491 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:33.223919 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f120d14f-a94b-442c-ad91-efe6215273f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:33.224537 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f120d14f-a94b-442c-ad91-efe6215273f8 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:33.224810 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f120d14f-a94b-442c-ad91-efe6215273f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:33.225096 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f120d14f-a94b-442c-ad91-efe6215273f8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:33.231707 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f120d14f-a94b-442c-ad91-efe6215273f8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:33.233124 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f120d14f-a94b-442c-ad91-efe6215273f8 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:33.243966 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-f120d14f-a94b-442c-ad91-efe6215273f8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:02:33.243966 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:33.243966 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:33.243966 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:33.243966 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:33.243966 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:33.243966 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:33.243966 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:33.243966 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:33.243966 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:33.243966 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:33.243966 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:33.243966 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 12:02:33.243966 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 12:02:33.243966 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 12:02:33.244890 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 12:02:33.244890 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:33.244890 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:33.244890 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:33.244890 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:33.244890 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:33.244890 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:33.244890 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:33.244890 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:33.244890 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:33.244890 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:33.244890 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:33.244890 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:33.244890 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:33.244890 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:33.244890 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:33.244890 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:33.245830 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 12:02:33.245830 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 12:02:33.245830 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 12:02:33.245830 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 12:02:33.245830 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 12:02:33.245830 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 12:02:33.245830 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:02:33.245830 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:02:33.247534 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 531/1072] 10.222.0.61 () {56 vars in 1111 bytes} [Fri Aug 26 12:02:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:02:33.250772 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87beb579-a301-4228-a788-e05ca86ddf42 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:33.355072 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87beb579-a301-4228-a788-e05ca86ddf42 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:33.390292 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7805d5b-09cb-4a26-9a0f-72a02b37ec7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:33.390904 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7805d5b-09cb-4a26-9a0f-72a02b37ec7b None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:33.391185 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7805d5b-09cb-4a26-9a0f-72a02b37ec7b None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:33.391478 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7805d5b-09cb-4a26-9a0f-72a02b37ec7b None admin] PATH_INFO: `/v3/projects` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:33.392164 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7805d5b-09cb-4a26-9a0f-72a02b37ec7b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:33.392467 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7805d5b-09cb-4a26-9a0f-72a02b37ec7b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:33.393886 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7805d5b-09cb-4a26-9a0f-72a02b37ec7b None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:33.406787 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 542/1073] 10.222.0.61 () {56 vars in 1148 bytes} [Fri Aug 26 12:02:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:02:33.412930 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f120d14f-a94b-442c-ad91-efe6215273f8 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:33.454319 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7805d5b-09cb-4a26-9a0f-72a02b37ec7b None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:33.507914 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:33.520255 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f120d14f-a94b-442c-ad91-efe6215273f8 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:33.542186 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:33.558378 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493ec726-0d65-40a2-9193-7f5e4f50dbe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:33.559597 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493ec726-0d65-40a2-9193-7f5e4f50dbe4 None admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:33.559893 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493ec726-0d65-40a2-9193-7f5e4f50dbe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:33.560241 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493ec726-0d65-40a2-9193-7f5e4f50dbe4 None admin] PATH_INFO: `/v3/projects/ecd68423a83a427681d44dbf266fce6e/users/7aacd7d6d17e4f10ba5c9107a12beafa/roles/ebae7fe0ca914cd1a0106219ed37c042` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:33.562359 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493ec726-0d65-40a2-9193-7f5e4f50dbe4 None admin] RBAC: Authorizing `identity:create_grant(project_id=ecd68423a83a427681d44dbf266fce6e, user_id=7aacd7d6d17e4f10ba5c9107a12beafa, role_id=ebae7fe0ca914cd1a0106219ed37c042)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:33.563839 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493ec726-0d65-40a2-9193-7f5e4f50dbe4 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:33.576872 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7805d5b-09cb-4a26-9a0f-72a02b37ec7b None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:33.577580 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 532/1074] 10.222.0.61 () {58 vars in 1439 bytes} [Fri Aug 26 12:02:33 2022] PUT /identity/v3/projects/ecd68423a83a427681d44dbf266fce6e/users/7aacd7d6d17e4f10ba5c9107a12beafa/roles/ebae7fe0ca914cd1a0106219ed37c042 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 12:02:33.624708 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14238916-3dcf-4b80-af83-d7742671d301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:33.625366 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14238916-3dcf-4b80-af83-d7742671d301 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:33.625643 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14238916-3dcf-4b80-af83-d7742671d301 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:33.625911 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14238916-3dcf-4b80-af83-d7742671d301 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:33.639384 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14238916-3dcf-4b80-af83-d7742671d301 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:02:33.644132 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-14238916-3dcf-4b80-af83-d7742671d301 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 12:02:33.644132 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:33.644132 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:33.644132 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:33.644132 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:33.644132 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:33.644132 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:33.644132 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:33.644132 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:33.644132 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:33.644132 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:33.644132 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:33.644132 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:02:33.644132 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:02:33.644132 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:02:33.644132 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:02:33.645181 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:02:33.645181 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:02:33.645181 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:02:33.645181 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:02:33.645181 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:33.645181 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:33.645181 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:33.645181 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:33.645181 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:33.645181 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:33.645181 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:33.645181 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:33.645181 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:33.645181 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:33.645181 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:33.645181 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:33.646043 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:33.646043 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:33.646043 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:33.646043 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:33.646043 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:02:33.646043 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:02:33.646043 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:02:33.646043 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:02:33.646043 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:02:33.646043 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:02:33.646043 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 12:02:33.646043 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:33.646043 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 543/1075] 10.222.0.61 () {56 vars in 1096 bytes} [Fri Aug 26 12:02:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 12:02:33.653105 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493ec726-0d65-40a2-9193-7f5e4f50dbe4 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:33.760355 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493ec726-0d65-40a2-9193-7f5e4f50dbe4 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:33.795670 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1048b06-2860-4178-a2d0-92dd45a80e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:33.796282 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1048b06-2860-4178-a2d0-92dd45a80e6d None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:33.796614 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1048b06-2860-4178-a2d0-92dd45a80e6d None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:33.796890 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1048b06-2860-4178-a2d0-92dd45a80e6d None admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:33.797546 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1048b06-2860-4178-a2d0-92dd45a80e6d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:33.797895 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1048b06-2860-4178-a2d0-92dd45a80e6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:33.799282 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1048b06-2860-4178-a2d0-92dd45a80e6d None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:33.812889 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 533/1076] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:02:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:33.815191 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14238916-3dcf-4b80-af83-d7742671d301 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:33.916664 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:33.921160 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14238916-3dcf-4b80-af83-d7742671d301 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:33.955728 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b76b904-df29-43a1-84b6-e5d7ce9fd844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:33.956353 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b76b904-df29-43a1-84b6-e5d7ce9fd844 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:33.956631 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b76b904-df29-43a1-84b6-e5d7ce9fd844 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:33.956902 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b76b904-df29-43a1-84b6-e5d7ce9fd844 None admin] PATH_INFO: `/v3/users/07ce16fc38a9414889c6c1b290622b79` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:33.958008 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b76b904-df29-43a1-84b6-e5d7ce9fd844 None admin] RBAC: Authorizing `identity:get_user(user_id=07ce16fc38a9414889c6c1b290622b79)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:33.959457 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b76b904-df29-43a1-84b6-e5d7ce9fd844 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:33.965683 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 544/1077] 10.222.0.61 () {56 vars in 1177 bytes} [Fri Aug 26 12:02:33 2022] GET /identity/v3/users/07ce16fc38a9414889c6c1b290622b79 => generated 285 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:33.966989 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1048b06-2860-4178-a2d0-92dd45a80e6d None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:34.012343 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:34.074223 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1048b06-2860-4178-a2d0-92dd45a80e6d None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:34.108313 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979c346e-8780-4441-b900-6fe0949bbade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:34.108935 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979c346e-8780-4441-b900-6fe0949bbade None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:34.109213 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979c346e-8780-4441-b900-6fe0949bbade None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:34.109485 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979c346e-8780-4441-b900-6fe0949bbade None admin] PATH_INFO: `/v3/projects/5367531f36164255a59bd78a9d4f1b0c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:34.110593 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979c346e-8780-4441-b900-6fe0949bbade None admin] RBAC: Authorizing `identity:get_project(project_id=5367531f36164255a59bd78a9d4f1b0c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:34.112043 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979c346e-8780-4441-b900-6fe0949bbade None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:34.118401 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 534/1078] 10.222.0.61 () {56 vars in 1186 bytes} [Fri Aug 26 12:02:33 2022] GET /identity/v3/projects/5367531f36164255a59bd78a9d4f1b0c => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:02:34.119928 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b76b904-df29-43a1-84b6-e5d7ce9fd844 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:34.227079 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b76b904-df29-43a1-84b6-e5d7ce9fd844 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:34.261442 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09619dc8-abed-4df2-9299-6e99008d18f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:34.262343 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09619dc8-abed-4df2-9299-6e99008d18f3 None admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:34.262631 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09619dc8-abed-4df2-9299-6e99008d18f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:34.262874 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09619dc8-abed-4df2-9299-6e99008d18f3 None admin] PATH_INFO: `/v3/projects/5367531f36164255a59bd78a9d4f1b0c/users/07ce16fc38a9414889c6c1b290622b79/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:34.264769 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09619dc8-abed-4df2-9299-6e99008d18f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=5367531f36164255a59bd78a9d4f1b0c, user_id=07ce16fc38a9414889c6c1b290622b79, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:34.266181 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09619dc8-abed-4df2-9299-6e99008d18f3 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:34.279784 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 545/1079] 10.222.0.61 () {58 vars in 1439 bytes} [Fri Aug 26 12:02:34 2022] PUT /identity/v3/projects/5367531f36164255a59bd78a9d4f1b0c/users/07ce16fc38a9414889c6c1b290622b79/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:02:34.379275 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ff1335-90d3-41d9-83ba-aae949921886 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:34.379628 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ff1335-90d3-41d9-83ba-aae949921886 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:34.379844 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ff1335-90d3-41d9-83ba-aae949921886 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:34.381375 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 535/1080] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:02:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:02:34.387822 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17855608-14d8-4ed7-9729-487ac6d757c4 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:34.388344 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17855608-14d8-4ed7-9729-487ac6d757c4 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:34.388344 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17855608-14d8-4ed7-9729-487ac6d757c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:34.407408 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-17855608-14d8-4ed7-9729-487ac6d757c4 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:02:34.408620 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-17855608-14d8-4ed7-9729-487ac6d757c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:02:34.435984 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:34.483996 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 546/1081] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:02:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 12:02:34.759481 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ff1335-90d3-41d9-83ba-aae949921886 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:34.864352 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ff1335-90d3-41d9-83ba-aae949921886 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:34.874827 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:34.900139 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f7974b-d57c-4952-9dee-2bbbd7b60c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:34.900745 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f7974b-d57c-4952-9dee-2bbbd7b60c8c None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:34.901021 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f7974b-d57c-4952-9dee-2bbbd7b60c8c None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:34.901293 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f7974b-d57c-4952-9dee-2bbbd7b60c8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:34.968946 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f7974b-d57c-4952-9dee-2bbbd7b60c8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:34.970393 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f7974b-d57c-4952-9dee-2bbbd7b60c8c None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:35.068325 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 536/1082] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:02:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 12:02:35.073871 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17855608-14d8-4ed7-9729-487ac6d757c4 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:35.175342 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17855608-14d8-4ed7-9729-487ac6d757c4 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:35.209956 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b184bf14-2be4-4eeb-8368-eeca11e08bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:35.210587 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b184bf14-2be4-4eeb-8368-eeca11e08bbe None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:35.210865 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b184bf14-2be4-4eeb-8368-eeca11e08bbe None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:35.211135 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b184bf14-2be4-4eeb-8368-eeca11e08bbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:35.221938 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b184bf14-2be4-4eeb-8368-eeca11e08bbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:35.223407 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b184bf14-2be4-4eeb-8368-eeca11e08bbe None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:35.238413 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-b184bf14-2be4-4eeb-8368-eeca11e08bbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:02:35.238413 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:35.238413 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:35.238413 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:35.238413 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:35.238413 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:35.238413 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:35.238413 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:35.238413 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:35.238413 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:35.238413 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:35.238413 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:35.238413 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:02:35.238413 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:02:35.238413 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:02:35.238413 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:02:35.239656 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:35.239656 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:35.239656 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:35.239656 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:35.239656 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:35.239656 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:35.239656 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:35.239656 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:35.239656 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:35.239656 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:35.239656 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:35.239656 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:35.239656 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:35.239656 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:35.239656 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:35.239656 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:35.239656 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:02:35.240849 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:02:35.240849 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:02:35.240849 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:35.243553 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 547/1083] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:02:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:02:35.245402 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f7974b-d57c-4952-9dee-2bbbd7b60c8c None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:35.257161 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:35.347190 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f7974b-d57c-4952-9dee-2bbbd7b60c8c None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:35.382144 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e292d6-8383-4146-a5c1-11dfcbe3aed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:35.382760 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e292d6-8383-4146-a5c1-11dfcbe3aed1 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:35.383036 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e292d6-8383-4146-a5c1-11dfcbe3aed1 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:35.383578 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e292d6-8383-4146-a5c1-11dfcbe3aed1 None admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:35.384222 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e292d6-8383-4146-a5c1-11dfcbe3aed1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:35.384523 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e292d6-8383-4146-a5c1-11dfcbe3aed1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:35.385892 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e292d6-8383-4146-a5c1-11dfcbe3aed1 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:35.398292 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 537/1084] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:02:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:02:35.526303 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c740a4ff-f69d-4cd5-bddd-567d073c97bf None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:35.526817 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c740a4ff-f69d-4cd5-bddd-567d073c97bf None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:35.526817 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c740a4ff-f69d-4cd5-bddd-567d073c97bf None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:35.529194 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 548/1085] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:02:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:02:35.534406 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4638d00-5afe-4edf-9e9d-662d5e3b03d5 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:35.534961 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4638d00-5afe-4edf-9e9d-662d5e3b03d5 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:35.535274 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4638d00-5afe-4edf-9e9d-662d5e3b03d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:35.554400 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-e4638d00-5afe-4edf-9e9d-662d5e3b03d5 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:02:35.555619 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-e4638d00-5afe-4edf-9e9d-662d5e3b03d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:02:35.623880 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:35.628981 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 538/1086] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:02:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 12:02:35.897146 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c740a4ff-f69d-4cd5-bddd-567d073c97bf None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:35.910371 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:35.998182 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c740a4ff-f69d-4cd5-bddd-567d073c97bf None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:36.030916 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22856bea-c5b7-4660-8522-0e9a7e76b542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:36.031620 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22856bea-c5b7-4660-8522-0e9a7e76b542 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:36.031897 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22856bea-c5b7-4660-8522-0e9a7e76b542 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:36.032165 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22856bea-c5b7-4660-8522-0e9a7e76b542 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:36.041780 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22856bea-c5b7-4660-8522-0e9a7e76b542 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:02:36.046975 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-22856bea-c5b7-4660-8522-0e9a7e76b542 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 12:02:36.046975 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:36.046975 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:36.046975 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:36.046975 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:36.046975 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:36.046975 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:36.046975 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:36.046975 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:36.046975 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:36.046975 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:36.046975 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:36.046975 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:02:36.046975 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:02:36.046975 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:02:36.046975 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:02:36.048122 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:02:36.048122 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:02:36.048122 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:02:36.048122 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:02:36.048122 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:36.048122 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:36.048122 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:36.048122 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:36.048122 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:36.048122 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:36.048122 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:36.048122 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:36.048122 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:36.048122 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:36.048122 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:36.048122 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:36.049268 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:36.049268 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:36.049268 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:36.049268 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:36.049268 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:02:36.049268 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:02:36.049268 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:02:36.049268 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:02:36.049268 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:02:36.049268 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:02:36.049268 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 26 12:02:36.049268 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:36.051518 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 549/1087] 10.222.0.61 () {56 vars in 1120 bytes} [Fri Aug 26 12:02:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:02:36.053898 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4638d00-5afe-4edf-9e9d-662d5e3b03d5 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:36.153686 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4638d00-5afe-4edf-9e9d-662d5e3b03d5 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:36.185776 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a160ab-f564-4c6e-9851-a26f51bd4f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:36.186368 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a160ab-f564-4c6e-9851-a26f51bd4f42 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:36.186640 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a160ab-f564-4c6e-9851-a26f51bd4f42 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:36.186912 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a160ab-f564-4c6e-9851-a26f51bd4f42 None admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:36.187584 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a160ab-f564-4c6e-9851-a26f51bd4f42 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:36.187881 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a160ab-f564-4c6e-9851-a26f51bd4f42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:36.189236 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a160ab-f564-4c6e-9851-a26f51bd4f42 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:36.202244 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 539/1088] 10.222.0.61 () {56 vars in 1115 bytes} [Fri Aug 26 12:02:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:36.203485 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22856bea-c5b7-4660-8522-0e9a7e76b542 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:36.235429 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e494b-4a5b-4ea3-81d0-f17da02b2c11 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:36.235711 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e494b-4a5b-4ea3-81d0-f17da02b2c11 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:36.236009 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e494b-4a5b-4ea3-81d0-f17da02b2c11 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:36.237350 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 540/1089] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:02:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 12:02:36.242253 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f813c-2c01-4dcd-8b9c-4c34667a6e48 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:36.242536 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f813c-2c01-4dcd-8b9c-4c34667a6e48 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:36.242807 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f813c-2c01-4dcd-8b9c-4c34667a6e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:36.260331 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-485f813c-2c01-4dcd-8b9c-4c34667a6e48 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:02:36.261509 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-485f813c-2c01-4dcd-8b9c-4c34667a6e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:02:36.274332 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:36.307288 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22856bea-c5b7-4660-8522-0e9a7e76b542 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:36.318548 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:36.330471 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 541/1090] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:02:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 12:02:36.342145 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d50be04-fc74-489e-8816-6ea7257312e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:36.342761 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50be04-fc74-489e-8816-6ea7257312e2 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:36.343040 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50be04-fc74-489e-8816-6ea7257312e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:36.343342 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50be04-fc74-489e-8816-6ea7257312e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:36.353081 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d50be04-fc74-489e-8816-6ea7257312e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:36.354496 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d50be04-fc74-489e-8816-6ea7257312e2 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:36.369055 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-1d50be04-fc74-489e-8816-6ea7257312e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:02:36.369055 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:36.369055 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:36.369055 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:36.369055 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:36.369055 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:36.369055 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:36.369055 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:36.369055 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:36.369055 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:36.369055 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:36.369055 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:36.369055 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:02:36.369055 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:02:36.369055 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:02:36.369055 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:02:36.370165 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:36.370165 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:36.370165 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:36.370165 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:36.370165 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:36.370165 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:36.370165 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:36.370165 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:36.370165 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:36.370165 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:36.370165 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:36.370165 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:36.370165 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:36.370165 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:36.370165 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:36.370165 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:36.370165 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:02:36.371075 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:02:36.371075 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:02:36.371075 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:36.371075 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 550/1091] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:02:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:02:36.375914 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f813c-2c01-4dcd-8b9c-4c34667a6e48 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:36.471577 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f813c-2c01-4dcd-8b9c-4c34667a6e48 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:36.504224 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d0a45d-ae86-422a-aee0-3fbaa2988358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:36.504818 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d0a45d-ae86-422a-aee0-3fbaa2988358 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:36.505095 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d0a45d-ae86-422a-aee0-3fbaa2988358 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:36.505388 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d0a45d-ae86-422a-aee0-3fbaa2988358 None admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:36.506017 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d0a45d-ae86-422a-aee0-3fbaa2988358 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:36.506314 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d0a45d-ae86-422a-aee0-3fbaa2988358 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:36.507708 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d0a45d-ae86-422a-aee0-3fbaa2988358 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:36.519444 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 542/1092] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:02:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:02:36.524982 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d50be04-fc74-489e-8816-6ea7257312e2 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:36.599171 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d0a45d-ae86-422a-aee0-3fbaa2988358 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:36.623073 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d50be04-fc74-489e-8816-6ea7257312e2 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:36.645327 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:36.654563 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:36.657711 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e359f68-eaf7-448e-91a0-8909a2dbb335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:36.658336 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e359f68-eaf7-448e-91a0-8909a2dbb335 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:36.658613 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e359f68-eaf7-448e-91a0-8909a2dbb335 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:36.658885 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e359f68-eaf7-448e-91a0-8909a2dbb335 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:36.666310 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e359f68-eaf7-448e-91a0-8909a2dbb335 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:36.667798 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e359f68-eaf7-448e-91a0-8909a2dbb335 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:36.678528 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-9e359f68-eaf7-448e-91a0-8909a2dbb335 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 26 12:02:36.678528 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:36.678528 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:36.678528 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:36.678528 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:36.678528 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:36.678528 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:36.678528 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:36.678528 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:36.678528 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:36.678528 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:36.678528 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:36.678528 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 12:02:36.678528 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 12:02:36.678528 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 12:02:36.678528 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 12:02:36.679631 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:36.679631 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:36.679631 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 12:02:36.679631 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:02:36.679631 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 12:02:36.679631 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:02:36.679631 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:36.679631 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:36.679631 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:36.679631 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:36.679631 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:36.679631 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:36.679631 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:36.679631 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:36.679631 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:36.679631 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:36.679631 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:36.679631 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:36.680602 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:36.680602 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:36.680602 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 12:02:36.680602 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 12:02:36.680602 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 12:02:36.680602 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 12:02:36.680602 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 12:02:36.680602 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 12:02:36.680602 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 26 12:02:36.680602 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:36.680602 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 551/1093] 10.222.0.61 () {56 vars in 1093 bytes} [Fri Aug 26 12:02:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 12:02:36.685913 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e359f68-eaf7-448e-91a0-8909a2dbb335 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:36.706707 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d0a45d-ae86-422a-aee0-3fbaa2988358 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:36.739106 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128a65e7-1c3d-470e-b20c-1f5157436be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:36.740090 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a65e7-1c3d-470e-b20c-1f5157436be3 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:36.740370 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a65e7-1c3d-470e-b20c-1f5157436be3 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:36.740647 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a65e7-1c3d-470e-b20c-1f5157436be3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:36.755672 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128a65e7-1c3d-470e-b20c-1f5157436be3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:02:36.760513 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-128a65e7-1c3d-470e-b20c-1f5157436be3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 12:02:36.760513 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:36.760513 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:36.760513 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:36.760513 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:36.760513 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:36.760513 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:36.760513 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:36.760513 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:36.760513 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:36.760513 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:36.760513 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:36.760513 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:02:36.760513 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:02:36.760513 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:02:36.760513 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:02:36.761478 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:02:36.761478 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:02:36.761478 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:02:36.761478 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:02:36.761478 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:36.761478 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:36.761478 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:36.761478 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:36.761478 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:36.761478 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:36.761478 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:36.761478 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:36.761478 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:36.761478 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:36.761478 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:36.761478 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:36.763024 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:36.763024 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:36.763024 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:36.763024 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:36.763024 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:02:36.763024 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:02:36.763024 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:02:36.763024 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:02:36.763024 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:02:36.763024 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:02:36.763024 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 12:02:36.763024 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:02:36.763024 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 543/1094] 10.222.0.61 () {56 vars in 1096 bytes} [Fri Aug 26 12:02:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 12:02:36.768081 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128a65e7-1c3d-470e-b20c-1f5157436be3 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:36.788639 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e359f68-eaf7-448e-91a0-8909a2dbb335 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:36.821712 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ed993-7872-4666-9be4-4a4b4137e96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:36.822318 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ed993-7872-4666-9be4-4a4b4137e96f None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:36.822599 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ed993-7872-4666-9be4-4a4b4137e96f None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:36.822873 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ed993-7872-4666-9be4-4a4b4137e96f None admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:36.823629 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07ed993-7872-4666-9be4-4a4b4137e96f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:36.823936 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07ed993-7872-4666-9be4-4a4b4137e96f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:36.826000 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07ed993-7872-4666-9be4-4a4b4137e96f None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:36.843428 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 552/1095] 10.222.0.61 () {56 vars in 1133 bytes} [Fri Aug 26 12:02:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:02:36.849100 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ed993-7872-4666-9be4-4a4b4137e96f None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:36.865074 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128a65e7-1c3d-470e-b20c-1f5157436be3 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:36.878075 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:36.890447 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:36.900341 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9ec6f-9031-4464-800e-5a86a0882a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:36.900937 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9ec6f-9031-4464-800e-5a86a0882a97 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:36.901211 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9ec6f-9031-4464-800e-5a86a0882a97 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:36.901478 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9ec6f-9031-4464-800e-5a86a0882a97 None admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:36.902124 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e9ec6f-9031-4464-800e-5a86a0882a97 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:36.903021 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e9ec6f-9031-4464-800e-5a86a0882a97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:36.904508 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e9ec6f-9031-4464-800e-5a86a0882a97 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:36.912791 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 544/1096] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:02:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:02:36.919267 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9ec6f-9031-4464-800e-5a86a0882a97 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:36.948760 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ed993-7872-4666-9be4-4a4b4137e96f None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:36.984271 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a2dc00-44f3-4c5e-a8bc-528432ea493c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:36.984871 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2dc00-44f3-4c5e-a8bc-528432ea493c None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:36.985151 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2dc00-44f3-4c5e-a8bc-528432ea493c None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:36.985430 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2dc00-44f3-4c5e-a8bc-528432ea493c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:37.016542 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9ec6f-9031-4464-800e-5a86a0882a97 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:37.049594 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8179cacb-cd6b-49e5-a4c8-84310bf4fecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:37.050211 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8179cacb-cd6b-49e5-a4c8-84310bf4fecc None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:37.050495 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8179cacb-cd6b-49e5-a4c8-84310bf4fecc None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:37.050772 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8179cacb-cd6b-49e5-a4c8-84310bf4fecc None admin] PATH_INFO: `/v3/users/07ce16fc38a9414889c6c1b290622b79` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:37.051459 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a2dc00-44f3-4c5e-a8bc-528432ea493c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:37.051915 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8179cacb-cd6b-49e5-a4c8-84310bf4fecc None admin] RBAC: Authorizing `identity:get_user(user_id=07ce16fc38a9414889c6c1b290622b79)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:37.052905 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a2dc00-44f3-4c5e-a8bc-528432ea493c None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:37.053322 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8179cacb-cd6b-49e5-a4c8-84310bf4fecc None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:37.055042 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 545/1097] 10.222.0.61 () {56 vars in 1177 bytes} [Fri Aug 26 12:02:36 2022] GET /identity/v3/users/07ce16fc38a9414889c6c1b290622b79 => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:02:37.060959 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8179cacb-cd6b-49e5-a4c8-84310bf4fecc None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:37.092958 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:37.103326 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:37.151789 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 553/1098] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:02:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 26 12:02:37.158058 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a2dc00-44f3-4c5e-a8bc-528432ea493c None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:37.158872 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8179cacb-cd6b-49e5-a4c8-84310bf4fecc None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:37.192220 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9df8ee-5fbf-45dd-b85d-a26c715a8ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:37.192821 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9df8ee-5fbf-45dd-b85d-a26c715a8ca0 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:37.193100 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9df8ee-5fbf-45dd-b85d-a26c715a8ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:37.193370 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9df8ee-5fbf-45dd-b85d-a26c715a8ca0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:37.254839 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a2dc00-44f3-4c5e-a8bc-528432ea493c None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:37.259843 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9df8ee-5fbf-45dd-b85d-a26c715a8ca0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:37.261280 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9df8ee-5fbf-45dd-b85d-a26c715a8ca0 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:37.289068 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310dcdf1-83a5-43db-a822-ab84b58ca02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:37.289617 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310dcdf1-83a5-43db-a822-ab84b58ca02a None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:37.289922 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310dcdf1-83a5-43db-a822-ab84b58ca02a None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:37.290197 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310dcdf1-83a5-43db-a822-ab84b58ca02a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:37.299500 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:37.302220 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310dcdf1-83a5-43db-a822-ab84b58ca02a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:37.302778 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:37.304064 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310dcdf1-83a5-43db-a822-ab84b58ca02a None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:37.319715 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-310dcdf1-83a5-43db-a822-ab84b58ca02a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:02:37.319715 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:37.319715 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:37.319715 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:37.319715 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:37.319715 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:37.319715 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:37.319715 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:37.319715 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:37.319715 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:37.319715 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:37.319715 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:37.319715 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:02:37.319715 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:02:37.319715 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:02:37.319715 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:02:37.320674 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:37.320674 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:37.320674 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:37.320674 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:37.320674 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:37.320674 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:37.320674 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:37.320674 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:37.320674 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:37.320674 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:37.320674 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:37.320674 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:37.320674 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:37.320674 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:37.320674 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:37.320674 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:37.320674 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:02:37.321598 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:02:37.321598 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:02:37.321598 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:37.321598 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 554/1099] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:02:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 12:02:37.327449 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310dcdf1-83a5-43db-a822-ab84b58ca02a None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:37.363379 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 546/1100] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:02:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 26 12:02:37.369270 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9df8ee-5fbf-45dd-b85d-a26c715a8ca0 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:37.427622 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310dcdf1-83a5-43db-a822-ab84b58ca02a None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:37.461284 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b034f9-fc6c-4194-8a59-263195c2d8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:37.461881 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b034f9-fc6c-4194-8a59-263195c2d8c9 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:37.462158 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b034f9-fc6c-4194-8a59-263195c2d8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:37.462434 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b034f9-fc6c-4194-8a59-263195c2d8c9 None admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:37.463067 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b034f9-fc6c-4194-8a59-263195c2d8c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:37.463405 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b034f9-fc6c-4194-8a59-263195c2d8c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:37.464764 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b034f9-fc6c-4194-8a59-263195c2d8c9 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:37.465141 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9df8ee-5fbf-45dd-b85d-a26c715a8ca0 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:37.481037 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 555/1101] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:02:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:02:37.482044 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b034f9-fc6c-4194-8a59-263195c2d8c9 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:37.499550 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e560a3-165b-422b-99c1-5c599eb9dae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:37.500180 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e560a3-165b-422b-99c1-5c599eb9dae9 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:37.500443 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e560a3-165b-422b-99c1-5c599eb9dae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:37.500708 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e560a3-165b-422b-99c1-5c599eb9dae9 None admin] PATH_INFO: `/v3/projects/5367531f36164255a59bd78a9d4f1b0c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:37.501796 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e560a3-165b-422b-99c1-5c599eb9dae9 None admin] RBAC: Authorizing `identity:get_project(project_id=5367531f36164255a59bd78a9d4f1b0c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:37.503236 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e560a3-165b-422b-99c1-5c599eb9dae9 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:37.504825 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 547/1102] 10.222.0.61 () {56 vars in 1186 bytes} [Fri Aug 26 12:02:37 2022] GET /identity/v3/projects/5367531f36164255a59bd78a9d4f1b0c => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:02:37.511006 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e560a3-165b-422b-99c1-5c599eb9dae9 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:37.523333 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:37.535922 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:37.583082 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b034f9-fc6c-4194-8a59-263195c2d8c9 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:37.609851 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e560a3-165b-422b-99c1-5c599eb9dae9 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:37.615800 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceb3216-01cc-4b81-8a02-f3652a3f7c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:37.616416 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb3216-01cc-4b81-8a02-f3652a3f7c12 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:37.616695 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb3216-01cc-4b81-8a02-f3652a3f7c12 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:37.616960 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb3216-01cc-4b81-8a02-f3652a3f7c12 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:37.624449 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceb3216-01cc-4b81-8a02-f3652a3f7c12 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:37.625869 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceb3216-01cc-4b81-8a02-f3652a3f7c12 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:37.637043 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-7ceb3216-01cc-4b81-8a02-f3652a3f7c12 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:02:37.637043 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:37.637043 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:37.637043 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:37.637043 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:37.637043 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:37.637043 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:37.637043 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:37.637043 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:37.637043 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:37.637043 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:37.637043 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:37.637043 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 12:02:37.637043 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 12:02:37.637043 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 12:02:37.637951 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 12:02:37.637951 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:37.637951 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:37.637951 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:37.637951 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:37.637951 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:37.637951 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:37.637951 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:37.637951 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:37.637951 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:37.637951 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:37.637951 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:37.637951 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:37.637951 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:37.637951 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:37.637951 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:37.637951 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:37.638848 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 12:02:37.638848 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 12:02:37.638848 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 12:02:37.638848 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 12:02:37.638848 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 12:02:37.638848 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 12:02:37.638848 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:02:37.638848 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:37.638848 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 556/1103] 10.222.0.61 () {56 vars in 1111 bytes} [Fri Aug 26 12:02:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:02:37.644032 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceb3216-01cc-4b81-8a02-f3652a3f7c12 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:37.645076 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de1e04a-50af-4fae-9c2f-0a9f855df1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:37.645672 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de1e04a-50af-4fae-9c2f-0a9f855df1e8 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:37.645948 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de1e04a-50af-4fae-9c2f-0a9f855df1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:37.646210 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de1e04a-50af-4fae-9c2f-0a9f855df1e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:37.646924 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de1e04a-50af-4fae-9c2f-0a9f855df1e8 None admin] RBAC: Adding query filter params (role.id=70ec1ba484714c8ca3adb8fe5845107c, scope.project.id=5367531f36164255a59bd78a9d4f1b0c, user.id=07ce16fc38a9414889c6c1b290622b79) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:37.647249 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de1e04a-50af-4fae-9c2f-0a9f855df1e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:37.648657 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de1e04a-50af-4fae-9c2f-0a9f855df1e8 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:37.654943 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 548/1104] 10.222.0.61 () {56 vars in 1374 bytes} [Fri Aug 26 12:02:37 2022] GET /identity/v3/role_assignments?user.id=07ce16fc38a9414889c6c1b290622b79&scope.project.id=5367531f36164255a59bd78a9d4f1b0c&role.id=70ec1ba484714c8ca3adb8fe5845107c => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:02:37.757814 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceb3216-01cc-4b81-8a02-f3652a3f7c12 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:37.771881 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:37.794617 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df0300a-9d07-4872-96fa-1cdc51a4c389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:37.795254 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df0300a-9d07-4872-96fa-1cdc51a4c389 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:37.795534 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df0300a-9d07-4872-96fa-1cdc51a4c389 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:37.795810 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df0300a-9d07-4872-96fa-1cdc51a4c389 None admin] PATH_INFO: `/v3/projects` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:37.796499 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df0300a-9d07-4872-96fa-1cdc51a4c389 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:37.796805 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df0300a-9d07-4872-96fa-1cdc51a4c389 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:37.798236 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df0300a-9d07-4872-96fa-1cdc51a4c389 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:37.812482 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 557/1105] 10.222.0.61 () {56 vars in 1148 bytes} [Fri Aug 26 12:02:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:02:37.818607 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de1e04a-50af-4fae-9c2f-0a9f855df1e8 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:37.895271 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:37.919980 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de1e04a-50af-4fae-9c2f-0a9f855df1e8 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:37.952670 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfb3dd6-6db5-46cf-ae68-ab30f85cb544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:37.953267 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb3dd6-6db5-46cf-ae68-ab30f85cb544 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:37.953543 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb3dd6-6db5-46cf-ae68-ab30f85cb544 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:37.953811 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb3dd6-6db5-46cf-ae68-ab30f85cb544 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:37.954523 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfb3dd6-6db5-46cf-ae68-ab30f85cb544 None admin] RBAC: Adding query filter params (role.id=ebae7fe0ca914cd1a0106219ed37c042, scope.project.id=ecd68423a83a427681d44dbf266fce6e, user.id=7aacd7d6d17e4f10ba5c9107a12beafa) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:37.954827 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfb3dd6-6db5-46cf-ae68-ab30f85cb544 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:37.956282 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfb3dd6-6db5-46cf-ae68-ab30f85cb544 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:37.961906 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 549/1106] 10.222.0.61 () {56 vars in 1374 bytes} [Fri Aug 26 12:02:37 2022] GET /identity/v3/role_assignments?user.id=7aacd7d6d17e4f10ba5c9107a12beafa&scope.project.id=ecd68423a83a427681d44dbf266fce6e&role.id=ebae7fe0ca914cd1a0106219ed37c042 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:39.595649 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349682c2-c157-4b48-926e-6deaed16cbdc None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:39.596175 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349682c2-c157-4b48-926e-6deaed16cbdc None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:39.596175 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349682c2-c157-4b48-926e-6deaed16cbdc None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:39.597516 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 558/1107] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:02:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:02:39.602840 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10dc453-39ac-4423-9b44-c250a82ae8e2 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:39.603281 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10dc453-39ac-4423-9b44-c250a82ae8e2 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:39.603420 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10dc453-39ac-4423-9b44-c250a82ae8e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:39.620835 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-b10dc453-39ac-4423-9b44-c250a82ae8e2 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:02:39.622544 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-b10dc453-39ac-4423-9b44-c250a82ae8e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:02:39.685770 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 550/1108] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:02:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 12:02:39.954910 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349682c2-c157-4b48-926e-6deaed16cbdc None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:40.053701 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349682c2-c157-4b48-926e-6deaed16cbdc None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:40.087374 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5039600-2add-469d-a45e-594dcbdaa7a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:40.087980 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5039600-2add-469d-a45e-594dcbdaa7a6 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:40.088253 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5039600-2add-469d-a45e-594dcbdaa7a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:40.088520 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5039600-2add-469d-a45e-594dcbdaa7a6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:40.089597 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5039600-2add-469d-a45e-594dcbdaa7a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:40.091731 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5039600-2add-469d-a45e-594dcbdaa7a6 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:40.093277 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 559/1109] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:02:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:40.099045 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10dc453-39ac-4423-9b44-c250a82ae8e2 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:40.166807 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:40.198787 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10dc453-39ac-4423-9b44-c250a82ae8e2 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:40.231125 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88be231e-aaca-4595-8447-1bcdcc3d27fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:40.231680 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be231e-aaca-4595-8447-1bcdcc3d27fd None admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:40.232051 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be231e-aaca-4595-8447-1bcdcc3d27fd None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:40.232330 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be231e-aaca-4595-8447-1bcdcc3d27fd None admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:40.232957 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88be231e-aaca-4595-8447-1bcdcc3d27fd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:40.234361 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88be231e-aaca-4595-8447-1bcdcc3d27fd None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:40.254731 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 551/1110] 10.222.0.61 () {60 vars in 1132 bytes} [Fri Aug 26 12:02:40 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:02:42.188212 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61871797-c823-48f7-b896-1fb871258a69 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:42.188791 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61871797-c823-48f7-b896-1fb871258a69 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:42.188791 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61871797-c823-48f7-b896-1fb871258a69 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:42.190065 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 560/1111] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:02:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:02:42.195144 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb7b4e5-efce-4cc3-bf84-8e9862a816b0 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:42.195597 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb7b4e5-efce-4cc3-bf84-8e9862a816b0 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:42.195781 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb7b4e5-efce-4cc3-bf84-8e9862a816b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:42.213189 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-4fb7b4e5-efce-4cc3-bf84-8e9862a816b0 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:02:42.214361 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-4fb7b4e5-efce-4cc3-bf84-8e9862a816b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:02:42.279924 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 552/1112] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:02:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 12:02:42.552432 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61871797-c823-48f7-b896-1fb871258a69 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:42.594513 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:42.650144 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61871797-c823-48f7-b896-1fb871258a69 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:42.682709 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fe682d-93e2-4f2d-89ac-93d5c95fac5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:42.683342 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe682d-93e2-4f2d-89ac-93d5c95fac5b None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:42.683643 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe682d-93e2-4f2d-89ac-93d5c95fac5b None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:42.683911 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe682d-93e2-4f2d-89ac-93d5c95fac5b None admin] PATH_INFO: `/v3/roles/a60f381814fe4c1da324d13c8e7d5d20` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:42.685274 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fe682d-93e2-4f2d-89ac-93d5c95fac5b None admin] RBAC: Authorizing `identity:get_role(role_id=a60f381814fe4c1da324d13c8e7d5d20)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:42.686627 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fe682d-93e2-4f2d-89ac-93d5c95fac5b None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:42.687952 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 561/1113] 10.222.0.61 () {56 vars in 1177 bytes} [Fri Aug 26 12:02:42 2022] GET /identity/v3/roles/a60f381814fe4c1da324d13c8e7d5d20 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:42.693697 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb7b4e5-efce-4cc3-bf84-8e9862a816b0 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:42.747563 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:42.792266 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb7b4e5-efce-4cc3-bf84-8e9862a816b0 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:42.826721 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606890be-5afc-4cda-b9be-b28b409ae2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:42.827296 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606890be-5afc-4cda-b9be-b28b409ae2c0 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:42.827579 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606890be-5afc-4cda-b9be-b28b409ae2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:42.827847 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606890be-5afc-4cda-b9be-b28b409ae2c0 None admin] PATH_INFO: `/v3/users/570624a0b26245f097446fed50defef4` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:42.840867 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606890be-5afc-4cda-b9be-b28b409ae2c0 None admin] RBAC: Authorizing `identity:get_user(user_id=570624a0b26245f097446fed50defef4)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:42.842287 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606890be-5afc-4cda-b9be-b28b409ae2c0 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:42.843876 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 553/1114] 10.222.0.61 () {56 vars in 1177 bytes} [Fri Aug 26 12:02:42 2022] GET /identity/v3/users/570624a0b26245f097446fed50defef4 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:42.849648 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fe682d-93e2-4f2d-89ac-93d5c95fac5b None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:42.945928 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fe682d-93e2-4f2d-89ac-93d5c95fac5b None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:42.955873 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:42.980926 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccbca3a-2ed8-4b4d-a4f9-73797e9c7948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:42.981507 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccbca3a-2ed8-4b4d-a4f9-73797e9c7948 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:42.981784 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccbca3a-2ed8-4b4d-a4f9-73797e9c7948 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:42.982054 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccbca3a-2ed8-4b4d-a4f9-73797e9c7948 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:43.050429 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccbca3a-2ed8-4b4d-a4f9-73797e9c7948 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:43.051945 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccbca3a-2ed8-4b4d-a4f9-73797e9c7948 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:43.147158 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 562/1115] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:02:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 12:02:43.153783 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606890be-5afc-4cda-b9be-b28b409ae2c0 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:43.248146 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606890be-5afc-4cda-b9be-b28b409ae2c0 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:43.260661 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:43.282250 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36872acb-57f2-4795-a97c-0472cb24dbc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:43.282857 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36872acb-57f2-4795-a97c-0472cb24dbc5 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:43.283130 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36872acb-57f2-4795-a97c-0472cb24dbc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:43.283430 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36872acb-57f2-4795-a97c-0472cb24dbc5 None admin] PATH_INFO: `/v3/projects/5367531f36164255a59bd78a9d4f1b0c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:43.284521 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36872acb-57f2-4795-a97c-0472cb24dbc5 None admin] RBAC: Authorizing `identity:get_project(project_id=5367531f36164255a59bd78a9d4f1b0c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:43.285932 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36872acb-57f2-4795-a97c-0472cb24dbc5 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:43.287547 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 554/1116] 10.222.0.61 () {56 vars in 1186 bytes} [Fri Aug 26 12:02:43 2022] GET /identity/v3/projects/5367531f36164255a59bd78a9d4f1b0c => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:43.293626 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccbca3a-2ed8-4b4d-a4f9-73797e9c7948 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:43.303462 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:43.389890 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccbca3a-2ed8-4b4d-a4f9-73797e9c7948 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:43.421835 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd94d4a-a616-4a97-9ef7-eea14d1411bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:43.422360 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd94d4a-a616-4a97-9ef7-eea14d1411bb None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:43.422634 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd94d4a-a616-4a97-9ef7-eea14d1411bb None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:43.422905 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd94d4a-a616-4a97-9ef7-eea14d1411bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:43.423644 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd94d4a-a616-4a97-9ef7-eea14d1411bb None admin] RBAC: Adding query filter params (role.id=a60f381814fe4c1da324d13c8e7d5d20, scope.project.id=5367531f36164255a59bd78a9d4f1b0c, user.id=570624a0b26245f097446fed50defef4) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:43.423946 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd94d4a-a616-4a97-9ef7-eea14d1411bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:43.425347 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd94d4a-a616-4a97-9ef7-eea14d1411bb None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:43.430464 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 563/1117] 10.222.0.61 () {56 vars in 1374 bytes} [Fri Aug 26 12:02:43 2022] GET /identity/v3/role_assignments?user.id=570624a0b26245f097446fed50defef4&scope.project.id=5367531f36164255a59bd78a9d4f1b0c&role.id=a60f381814fe4c1da324d13c8e7d5d20 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:45.302414 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e2603-d91a-49a1-93d4-e0143783af6b None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:45.302955 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e2603-d91a-49a1-93d4-e0143783af6b None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:45.302955 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e2603-d91a-49a1-93d4-e0143783af6b None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:45.304322 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 555/1118] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:02:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:02:45.309466 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c9d75e-476c-4cbd-948e-a71cf6b9faf2 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:45.309882 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c9d75e-476c-4cbd-948e-a71cf6b9faf2 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:45.310020 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c9d75e-476c-4cbd-948e-a71cf6b9faf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:45.334808 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-54c9d75e-476c-4cbd-948e-a71cf6b9faf2 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:02:45.336006 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-54c9d75e-476c-4cbd-948e-a71cf6b9faf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:02:45.401021 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 564/1119] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:02:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 12:02:45.688680 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73e2603-d91a-49a1-93d4-e0143783af6b None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:45.789082 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73e2603-d91a-49a1-93d4-e0143783af6b None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:45.822895 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bdd956-c160-42fe-80fd-9adb3ea6dadf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:45.823535 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bdd956-c160-42fe-80fd-9adb3ea6dadf None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:45.823818 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bdd956-c160-42fe-80fd-9adb3ea6dadf None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:45.824111 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bdd956-c160-42fe-80fd-9adb3ea6dadf None admin] PATH_INFO: `/v3/roles/a60f381814fe4c1da324d13c8e7d5d20` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:45.825739 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bdd956-c160-42fe-80fd-9adb3ea6dadf None admin] RBAC: Authorizing `identity:get_role(role_id=a60f381814fe4c1da324d13c8e7d5d20)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:45.827097 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bdd956-c160-42fe-80fd-9adb3ea6dadf None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:45.828438 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 556/1120] 10.222.0.61 () {56 vars in 1177 bytes} [Fri Aug 26 12:02:45 2022] GET /identity/v3/roles/a60f381814fe4c1da324d13c8e7d5d20 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:45.834158 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c9d75e-476c-4cbd-948e-a71cf6b9faf2 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:45.836362 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:45.938049 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c9d75e-476c-4cbd-948e-a71cf6b9faf2 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:45.970906 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3925ed63-dea9-48d0-bd5d-c92c4c9900ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:45.971487 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3925ed63-dea9-48d0-bd5d-c92c4c9900ce None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:45.971768 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3925ed63-dea9-48d0-bd5d-c92c4c9900ce None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:45.972040 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3925ed63-dea9-48d0-bd5d-c92c4c9900ce None admin] PATH_INFO: `/v3/users/570624a0b26245f097446fed50defef4` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:45.973148 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3925ed63-dea9-48d0-bd5d-c92c4c9900ce None admin] RBAC: Authorizing `identity:get_user(user_id=570624a0b26245f097446fed50defef4)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:45.974566 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3925ed63-dea9-48d0-bd5d-c92c4c9900ce None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:45.976180 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 565/1121] 10.222.0.61 () {56 vars in 1177 bytes} [Fri Aug 26 12:02:45 2022] GET /identity/v3/users/570624a0b26245f097446fed50defef4 => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:45.982586 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bdd956-c160-42fe-80fd-9adb3ea6dadf None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:46.031094 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:46.081407 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bdd956-c160-42fe-80fd-9adb3ea6dadf None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:46.115483 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e40222-2e0d-4012-8333-d8cc5b0248c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:46.116104 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e40222-2e0d-4012-8333-d8cc5b0248c6 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:46.116384 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e40222-2e0d-4012-8333-d8cc5b0248c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:46.116657 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e40222-2e0d-4012-8333-d8cc5b0248c6 None admin] PATH_INFO: `/v3/projects/5367531f36164255a59bd78a9d4f1b0c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:46.117757 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e40222-2e0d-4012-8333-d8cc5b0248c6 None admin] RBAC: Authorizing `identity:get_project(project_id=5367531f36164255a59bd78a9d4f1b0c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:46.119179 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e40222-2e0d-4012-8333-d8cc5b0248c6 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:46.120829 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 557/1122] 10.222.0.61 () {56 vars in 1186 bytes} [Fri Aug 26 12:02:45 2022] GET /identity/v3/projects/5367531f36164255a59bd78a9d4f1b0c => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:46.127508 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3925ed63-dea9-48d0-bd5d-c92c4c9900ce None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:46.196377 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:46.227764 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3925ed63-dea9-48d0-bd5d-c92c4c9900ce None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:46.261576 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f88dc-2bda-4d6b-a9a3-2216886926d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:46.262497 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f88dc-2bda-4d6b-a9a3-2216886926d9 None admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:46.262785 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f88dc-2bda-4d6b-a9a3-2216886926d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:46.263060 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f88dc-2bda-4d6b-a9a3-2216886926d9 None admin] PATH_INFO: `/v3/projects/5367531f36164255a59bd78a9d4f1b0c/users/570624a0b26245f097446fed50defef4/roles/a60f381814fe4c1da324d13c8e7d5d20` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:46.265204 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2f88dc-2bda-4d6b-a9a3-2216886926d9 None admin] RBAC: Authorizing `identity:create_grant(project_id=5367531f36164255a59bd78a9d4f1b0c, user_id=570624a0b26245f097446fed50defef4, role_id=a60f381814fe4c1da324d13c8e7d5d20)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:46.266627 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2f88dc-2bda-4d6b-a9a3-2216886926d9 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:46.279356 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 566/1123] 10.222.0.61 () {58 vars in 1439 bytes} [Fri Aug 26 12:02:46 2022] PUT /identity/v3/projects/5367531f36164255a59bd78a9d4f1b0c/users/570624a0b26245f097446fed50defef4/roles/a60f381814fe4c1da324d13c8e7d5d20 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:02:48.128680 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f95ed32-64a0-41e1-8431-95e187d0f540 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:48.129235 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f95ed32-64a0-41e1-8431-95e187d0f540 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:48.129235 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f95ed32-64a0-41e1-8431-95e187d0f540 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:48.130542 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 558/1124] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:02:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:02:48.135915 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e860f8-00d5-4281-aee2-59efffba5ab8 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:48.136368 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e860f8-00d5-4281-aee2-59efffba5ab8 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:48.137174 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e860f8-00d5-4281-aee2-59efffba5ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:48.155728 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-c4e860f8-00d5-4281-aee2-59efffba5ab8 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:02:48.156897 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-c4e860f8-00d5-4281-aee2-59efffba5ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:02:48.220620 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 567/1125] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:02:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 12:02:48.489364 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f95ed32-64a0-41e1-8431-95e187d0f540 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:48.587364 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f95ed32-64a0-41e1-8431-95e187d0f540 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:48.602695 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:48.619618 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0effd07-e040-4518-891b-149fe72b2d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:48.620225 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0effd07-e040-4518-891b-149fe72b2d35 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:48.620503 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0effd07-e040-4518-891b-149fe72b2d35 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:48.620772 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0effd07-e040-4518-891b-149fe72b2d35 None admin] PATH_INFO: `/v3/roles/a60f381814fe4c1da324d13c8e7d5d20` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:48.622629 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0effd07-e040-4518-891b-149fe72b2d35 None admin] RBAC: Authorizing `identity:get_role(role_id=a60f381814fe4c1da324d13c8e7d5d20)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:48.624008 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0effd07-e040-4518-891b-149fe72b2d35 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:48.625296 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 559/1126] 10.222.0.61 () {56 vars in 1177 bytes} [Fri Aug 26 12:02:48 2022] GET /identity/v3/roles/a60f381814fe4c1da324d13c8e7d5d20 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:48.630963 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e860f8-00d5-4281-aee2-59efffba5ab8 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:48.687508 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:48.726947 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e860f8-00d5-4281-aee2-59efffba5ab8 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:48.761470 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ddbd98-2b34-4146-9a74-052d7cc02fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:48.762770 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ddbd98-2b34-4146-9a74-052d7cc02fec None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:48.763043 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ddbd98-2b34-4146-9a74-052d7cc02fec None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:48.763404 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ddbd98-2b34-4146-9a74-052d7cc02fec None admin] PATH_INFO: `/v3/users/570624a0b26245f097446fed50defef4` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:48.764507 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ddbd98-2b34-4146-9a74-052d7cc02fec None admin] RBAC: Authorizing `identity:get_user(user_id=570624a0b26245f097446fed50defef4)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:48.765918 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ddbd98-2b34-4146-9a74-052d7cc02fec None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:48.767524 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 568/1127] 10.222.0.61 () {56 vars in 1177 bytes} [Fri Aug 26 12:02:48 2022] GET /identity/v3/users/570624a0b26245f097446fed50defef4 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:48.773129 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0effd07-e040-4518-891b-149fe72b2d35 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:48.869387 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0effd07-e040-4518-891b-149fe72b2d35 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:48.902593 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ed7b25-9003-4e5c-8c9f-09222eb7a288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:48.903178 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ed7b25-9003-4e5c-8c9f-09222eb7a288 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:48.903488 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ed7b25-9003-4e5c-8c9f-09222eb7a288 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:48.903763 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ed7b25-9003-4e5c-8c9f-09222eb7a288 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:48.954047 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:48.969620 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ed7b25-9003-4e5c-8c9f-09222eb7a288 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:48.971069 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ed7b25-9003-4e5c-8c9f-09222eb7a288 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:49.066458 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 560/1128] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:02:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 12:02:49.072315 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ddbd98-2b34-4146-9a74-052d7cc02fec None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:49.168972 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ddbd98-2b34-4146-9a74-052d7cc02fec None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:49.194156 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:49.202186 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975ec61c-b9f1-46c2-ad6d-27db89ae819b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:49.202798 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ec61c-b9f1-46c2-ad6d-27db89ae819b None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:49.203073 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ec61c-b9f1-46c2-ad6d-27db89ae819b None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:49.203371 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ec61c-b9f1-46c2-ad6d-27db89ae819b None admin] PATH_INFO: `/v3/projects/5367531f36164255a59bd78a9d4f1b0c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:49.204446 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975ec61c-b9f1-46c2-ad6d-27db89ae819b None admin] RBAC: Authorizing `identity:get_project(project_id=5367531f36164255a59bd78a9d4f1b0c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:49.205856 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975ec61c-b9f1-46c2-ad6d-27db89ae819b None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:49.207467 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 569/1129] 10.222.0.61 () {56 vars in 1186 bytes} [Fri Aug 26 12:02:49 2022] GET /identity/v3/projects/5367531f36164255a59bd78a9d4f1b0c => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:49.213494 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ed7b25-9003-4e5c-8c9f-09222eb7a288 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:49.294520 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:49.310412 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ed7b25-9003-4e5c-8c9f-09222eb7a288 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:49.342931 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ea8319-0a35-4005-9fbb-12030fcfd1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:49.343486 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ea8319-0a35-4005-9fbb-12030fcfd1fd None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:49.343762 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ea8319-0a35-4005-9fbb-12030fcfd1fd None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:49.344027 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ea8319-0a35-4005-9fbb-12030fcfd1fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:49.344735 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ea8319-0a35-4005-9fbb-12030fcfd1fd None admin] RBAC: Adding query filter params (role.id=a60f381814fe4c1da324d13c8e7d5d20, scope.project.id=5367531f36164255a59bd78a9d4f1b0c, user.id=570624a0b26245f097446fed50defef4) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:49.345036 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ea8319-0a35-4005-9fbb-12030fcfd1fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:49.346443 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ea8319-0a35-4005-9fbb-12030fcfd1fd None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:49.352015 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 561/1130] 10.222.0.61 () {56 vars in 1374 bytes} [Fri Aug 26 12:02:49 2022] GET /identity/v3/role_assignments?user.id=570624a0b26245f097446fed50defef4&scope.project.id=5367531f36164255a59bd78a9d4f1b0c&role.id=a60f381814fe4c1da324d13c8e7d5d20 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:51.262844 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae29fbe-cf51-424c-a4e8-3d24cbd8bcd6 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:51.263441 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae29fbe-cf51-424c-a4e8-3d24cbd8bcd6 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:51.263441 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae29fbe-cf51-424c-a4e8-3d24cbd8bcd6 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:51.264847 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 570/1131] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:02:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:02:51.270024 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7103fd54-d448-47d0-b7de-6d711d93a5e0 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:51.270418 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7103fd54-d448-47d0-b7de-6d711d93a5e0 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:51.270565 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7103fd54-d448-47d0-b7de-6d711d93a5e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:51.288540 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-7103fd54-d448-47d0-b7de-6d711d93a5e0 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:02:51.289729 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-7103fd54-d448-47d0-b7de-6d711d93a5e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:02:51.354145 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 562/1132] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:02:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 12:02:51.625155 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae29fbe-cf51-424c-a4e8-3d24cbd8bcd6 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:51.724455 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae29fbe-cf51-424c-a4e8-3d24cbd8bcd6 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:51.759836 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96addfc7-679f-41f6-a150-3b7d05eea03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:51.760457 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96addfc7-679f-41f6-a150-3b7d05eea03a None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:51.760735 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96addfc7-679f-41f6-a150-3b7d05eea03a None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:51.761005 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96addfc7-679f-41f6-a150-3b7d05eea03a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:51.762316 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96addfc7-679f-41f6-a150-3b7d05eea03a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:51.763786 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96addfc7-679f-41f6-a150-3b7d05eea03a None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:51.765345 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 571/1133] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:02:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:51.770991 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7103fd54-d448-47d0-b7de-6d711d93a5e0 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:51.847236 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:51.868411 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7103fd54-d448-47d0-b7de-6d711d93a5e0 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:51.901401 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af76d689-b726-4b99-bcac-293fd47198ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:51.901936 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76d689-b726-4b99-bcac-293fd47198ab None admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:51.902813 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76d689-b726-4b99-bcac-293fd47198ab None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:51.903088 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76d689-b726-4b99-bcac-293fd47198ab None admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:51.903761 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af76d689-b726-4b99-bcac-293fd47198ab None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:51.905187 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af76d689-b726-4b99-bcac-293fd47198ab None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:51.930478 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 563/1134] 10.222.0.61 () {60 vars in 1141 bytes} [Fri Aug 26 12:02:51 2022] POST /identity/v3/projects => generated 311 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:02:53.858003 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc12ccf5-5b0e-4103-92c9-85070dd97ee3 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:53.858543 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc12ccf5-5b0e-4103-92c9-85070dd97ee3 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:53.858543 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc12ccf5-5b0e-4103-92c9-85070dd97ee3 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:53.859899 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 572/1135] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:02:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:02:53.864977 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46d92f0-54f6-4b24-b1c2-f443653c3abc None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:53.865343 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46d92f0-54f6-4b24-b1c2-f443653c3abc None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:53.865534 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46d92f0-54f6-4b24-b1c2-f443653c3abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:53.883898 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-f46d92f0-54f6-4b24-b1c2-f443653c3abc None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:02:53.885074 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-f46d92f0-54f6-4b24-b1c2-f443653c3abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:02:53.950631 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 564/1136] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:02:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 12:02:54.224845 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc12ccf5-5b0e-4103-92c9-85070dd97ee3 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:54.286967 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:54.325130 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc12ccf5-5b0e-4103-92c9-85070dd97ee3 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:54.359245 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0b80d3-9733-4fac-8011-238c8fe9d520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:54.359862 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b80d3-9733-4fac-8011-238c8fe9d520 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:54.360140 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b80d3-9733-4fac-8011-238c8fe9d520 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:54.360414 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b80d3-9733-4fac-8011-238c8fe9d520 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:54.361518 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0b80d3-9733-4fac-8011-238c8fe9d520 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:54.362944 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0b80d3-9733-4fac-8011-238c8fe9d520 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:54.364520 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 573/1137] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:02:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:54.370396 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46d92f0-54f6-4b24-b1c2-f443653c3abc None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:54.426626 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:54.467302 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46d92f0-54f6-4b24-b1c2-f443653c3abc None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:54.500299 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f788a263-fcb8-4b82-afce-df0f7ddcbb01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:54.500827 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f788a263-fcb8-4b82-afce-df0f7ddcbb01 None admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:54.501100 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f788a263-fcb8-4b82-afce-df0f7ddcbb01 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:54.501365 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f788a263-fcb8-4b82-afce-df0f7ddcbb01 None admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:54.501984 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f788a263-fcb8-4b82-afce-df0f7ddcbb01 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:54.503423 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f788a263-fcb8-4b82-afce-df0f7ddcbb01 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:54.523509 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 565/1138] 10.222.0.61 () {60 vars in 1132 bytes} [Fri Aug 26 12:02:54 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:02:56.461708 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e447f4-d0f1-4ddf-8e05-5ae8c1e5c7b7 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:56.462260 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e447f4-d0f1-4ddf-8e05-5ae8c1e5c7b7 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:56.462260 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e447f4-d0f1-4ddf-8e05-5ae8c1e5c7b7 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:56.463610 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 574/1139] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:02:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:02:56.468927 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91676a52-2a69-4213-a07d-d76609da43b2 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:56.469319 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91676a52-2a69-4213-a07d-d76609da43b2 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:56.469464 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91676a52-2a69-4213-a07d-d76609da43b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:56.487257 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-91676a52-2a69-4213-a07d-d76609da43b2 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:02:56.488430 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-91676a52-2a69-4213-a07d-d76609da43b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:02:56.552706 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 566/1140] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:02:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 12:02:56.824208 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e447f4-d0f1-4ddf-8e05-5ae8c1e5c7b7 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:56.923246 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e447f4-d0f1-4ddf-8e05-5ae8c1e5c7b7 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:56.951022 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:56.956090 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecdb6f7-bd87-41df-ae53-3b9647d78435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:56.956702 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecdb6f7-bd87-41df-ae53-3b9647d78435 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:56.956976 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecdb6f7-bd87-41df-ae53-3b9647d78435 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:56.957243 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecdb6f7-bd87-41df-ae53-3b9647d78435 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:56.967574 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecdb6f7-bd87-41df-ae53-3b9647d78435 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:02:56.972126 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-fecdb6f7-bd87-41df-ae53-3b9647d78435 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 12:02:56.972126 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:02:56.972126 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:02:56.972126 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:02:56.972126 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:02:56.972126 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:02:56.972126 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:02:56.972126 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:02:56.972126 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:02:56.972126 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:02:56.972126 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:02:56.972126 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:02:56.972126 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:02:56.972126 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:02:56.972126 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:02:56.972126 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:02:56.973305 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:02:56.973305 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:02:56.973305 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:02:56.973305 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:02:56.973305 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:02:56.973305 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:02:56.973305 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:02:56.973305 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:02:56.973305 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:02:56.973305 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:02:56.973305 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:02:56.973305 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:02:56.973305 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:02:56.973305 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:02:56.973305 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:02:56.973305 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:02:56.974400 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:02:56.974400 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:02:56.974400 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:02:56.974400 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:02:56.974400 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:02:56.974400 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:02:56.974400 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:02:56.974400 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:02:56.974400 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:02:56.974400 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:02:56.974400 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 12:02:56.974400 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:02:56.974400 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 575/1141] 10.222.0.61 () {56 vars in 1096 bytes} [Fri Aug 26 12:02:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:02:56.979096 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91676a52-2a69-4213-a07d-d76609da43b2 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:57.023462 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:57.074875 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91676a52-2a69-4213-a07d-d76609da43b2 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:57.107398 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749c2ae4-2353-46cd-a091-7dc7ba4f6fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:57.107987 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749c2ae4-2353-46cd-a091-7dc7ba4f6fee None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:57.108262 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749c2ae4-2353-46cd-a091-7dc7ba4f6fee None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:57.108528 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749c2ae4-2353-46cd-a091-7dc7ba4f6fee None admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:57.109173 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749c2ae4-2353-46cd-a091-7dc7ba4f6fee None admin] RBAC: Adding query filter params (name=admin) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:57.109468 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749c2ae4-2353-46cd-a091-7dc7ba4f6fee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:57.110815 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749c2ae4-2353-46cd-a091-7dc7ba4f6fee None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:57.118442 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 567/1142] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:02:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:57.124453 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecdb6f7-bd87-41df-ae53-3b9647d78435 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:57.222601 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecdb6f7-bd87-41df-ae53-3b9647d78435 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:57.254807 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fda3461-d7f3-4608-ba7c-79a2d36d66f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:57.255448 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda3461-d7f3-4608-ba7c-79a2d36d66f0 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:57.255728 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda3461-d7f3-4608-ba7c-79a2d36d66f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:57.255999 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda3461-d7f3-4608-ba7c-79a2d36d66f0 None admin] PATH_INFO: `/v3/users/d8c4341f54a041e79f3c3e65f708dbed` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:57.269347 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fda3461-d7f3-4608-ba7c-79a2d36d66f0 None admin] RBAC: Authorizing `identity:get_user(user_id=d8c4341f54a041e79f3c3e65f708dbed)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:57.270774 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fda3461-d7f3-4608-ba7c-79a2d36d66f0 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:57.272373 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 576/1143] 10.222.0.61 () {56 vars in 1177 bytes} [Fri Aug 26 12:02:57 2022] GET /identity/v3/users/d8c4341f54a041e79f3c3e65f708dbed => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:57.278967 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749c2ae4-2353-46cd-a091-7dc7ba4f6fee None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:57.377247 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749c2ae4-2353-46cd-a091-7dc7ba4f6fee None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:57.386232 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:57.412245 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b178e07-4e16-4c65-8104-e76437607afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:57.412844 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b178e07-4e16-4c65-8104-e76437607afd None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:57.413123 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b178e07-4e16-4c65-8104-e76437607afd None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:57.413396 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b178e07-4e16-4c65-8104-e76437607afd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:57.481596 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b178e07-4e16-4c65-8104-e76437607afd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:57.483049 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b178e07-4e16-4c65-8104-e76437607afd None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:57.584738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 568/1144] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:02:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 12:02:57.590995 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fda3461-d7f3-4608-ba7c-79a2d36d66f0 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:57.649799 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:57.706593 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fda3461-d7f3-4608-ba7c-79a2d36d66f0 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:57.750644 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596d033e-e840-4d11-a71d-7b9584332fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:57.751683 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d033e-e840-4d11-a71d-7b9584332fc8 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:57.752150 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d033e-e840-4d11-a71d-7b9584332fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:57.752607 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d033e-e840-4d11-a71d-7b9584332fc8 None admin] PATH_INFO: `/v3/projects/06d12a8c45ae46709be68115b9e058f9` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:57.754361 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596d033e-e840-4d11-a71d-7b9584332fc8 None admin] RBAC: Authorizing `identity:get_project(project_id=06d12a8c45ae46709be68115b9e058f9)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:57.756559 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596d033e-e840-4d11-a71d-7b9584332fc8 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:57.758463 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 577/1145] 10.222.0.61 () {56 vars in 1186 bytes} [Fri Aug 26 12:02:57 2022] GET /identity/v3/projects/06d12a8c45ae46709be68115b9e058f9 => generated 311 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:57.766486 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b178e07-4e16-4c65-8104-e76437607afd None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:02:57.776717 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:02:57.866036 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b178e07-4e16-4c65-8104-e76437607afd None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:02:57.901913 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5568683b-e59b-4240-bc8c-d43e820a8bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:02:57.902559 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5568683b-e59b-4240-bc8c-d43e820a8bc6 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:57.902910 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5568683b-e59b-4240-bc8c-d43e820a8bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:57.903182 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5568683b-e59b-4240-bc8c-d43e820a8bc6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:57.903937 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5568683b-e59b-4240-bc8c-d43e820a8bc6 None admin] RBAC: Adding query filter params (role.id=70ec1ba484714c8ca3adb8fe5845107c, scope.project.id=06d12a8c45ae46709be68115b9e058f9, user.id=d8c4341f54a041e79f3c3e65f708dbed) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:02:57.904234 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5568683b-e59b-4240-bc8c-d43e820a8bc6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:02:57.905690 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5568683b-e59b-4240-bc8c-d43e820a8bc6 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:02:57.911085 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 569/1146] 10.222.0.61 () {56 vars in 1374 bytes} [Fri Aug 26 12:02:57 2022] GET /identity/v3/role_assignments?user.id=d8c4341f54a041e79f3c3e65f708dbed&scope.project.id=06d12a8c45ae46709be68115b9e058f9&role.id=70ec1ba484714c8ca3adb8fe5845107c => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:02:59.799261 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870893c-e89b-4812-8d2f-5a8cd86d0dcf None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:59.799721 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870893c-e89b-4812-8d2f-5a8cd86d0dcf None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:59.799868 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870893c-e89b-4812-8d2f-5a8cd86d0dcf None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:59.801145 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 578/1147] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:02:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:02:59.806132 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71201650-e706-4309-a6d5-95e466daf223 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:02:59.806556 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71201650-e706-4309-a6d5-95e466daf223 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:02:59.806688 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71201650-e706-4309-a6d5-95e466daf223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:02:59.826183 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-71201650-e706-4309-a6d5-95e466daf223 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:02:59.827403 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-71201650-e706-4309-a6d5-95e466daf223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:02:59.891896 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 570/1148] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:02:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 12:03:00.158881 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c870893c-e89b-4812-8d2f-5a8cd86d0dcf None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:03:00.259710 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c870893c-e89b-4812-8d2f-5a8cd86d0dcf None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:03:00.277000 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:03:00.294823 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06323469-b900-4d66-a25b-7f4429b6b6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:03:00.295474 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06323469-b900-4d66-a25b-7f4429b6b6eb None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:03:00.295755 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06323469-b900-4d66-a25b-7f4429b6b6eb None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:03:00.296025 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06323469-b900-4d66-a25b-7f4429b6b6eb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:03:00.305790 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06323469-b900-4d66-a25b-7f4429b6b6eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:03:00.311029 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-06323469-b900-4d66-a25b-7f4429b6b6eb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 12:03:00.311029 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:03:00.311029 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:03:00.311029 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:03:00.311029 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:03:00.311029 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:03:00.311029 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:03:00.311029 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:03:00.311029 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:03:00.311029 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:03:00.311029 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:03:00.311029 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:03:00.311029 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:03:00.311029 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:03:00.311029 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:03:00.311029 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:03:00.312152 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:03:00.312152 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:03:00.312152 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:03:00.312152 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:03:00.312152 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:03:00.312152 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:03:00.312152 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:03:00.312152 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:03:00.312152 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:03:00.312152 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:03:00.312152 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:03:00.312152 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:03:00.312152 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:03:00.312152 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:03:00.312152 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:03:00.312152 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:03:00.313345 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:03:00.313345 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:03:00.313345 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:03:00.313345 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:03:00.313345 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:03:00.313345 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:03:00.313345 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:03:00.313345 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:03:00.313345 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:03:00.313345 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:03:00.313345 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 12:03:00.313345 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:03:00.313345 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 579/1149] 10.222.0.61 () {56 vars in 1096 bytes} [Fri Aug 26 12:03:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:03:00.318148 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71201650-e706-4309-a6d5-95e466daf223 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:03:00.320344 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:03:00.414130 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71201650-e706-4309-a6d5-95e466daf223 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:03:00.447023 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2087bfc6-77f0-4e46-bf81-1131c18399f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:03:00.447635 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2087bfc6-77f0-4e46-bf81-1131c18399f0 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:03:00.447915 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2087bfc6-77f0-4e46-bf81-1131c18399f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:03:00.448186 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2087bfc6-77f0-4e46-bf81-1131c18399f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:03:00.448857 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2087bfc6-77f0-4e46-bf81-1131c18399f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:03:00.449158 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2087bfc6-77f0-4e46-bf81-1131c18399f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:03:00.450507 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2087bfc6-77f0-4e46-bf81-1131c18399f0 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:03:00.458167 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 571/1150] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:03:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:03:00.463963 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06323469-b900-4d66-a25b-7f4429b6b6eb None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:03:00.561954 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06323469-b900-4d66-a25b-7f4429b6b6eb None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:03:00.594426 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70aec62-b29b-48fa-83fb-0a686dea50d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:03:00.595034 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70aec62-b29b-48fa-83fb-0a686dea50d9 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:03:00.595344 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70aec62-b29b-48fa-83fb-0a686dea50d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:03:00.595617 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70aec62-b29b-48fa-83fb-0a686dea50d9 None admin] PATH_INFO: `/v3/users/d8c4341f54a041e79f3c3e65f708dbed` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:03:00.596717 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70aec62-b29b-48fa-83fb-0a686dea50d9 None admin] RBAC: Authorizing `identity:get_user(user_id=d8c4341f54a041e79f3c3e65f708dbed)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:03:00.598135 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70aec62-b29b-48fa-83fb-0a686dea50d9 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:03:00.599726 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 580/1151] 10.222.0.61 () {56 vars in 1177 bytes} [Fri Aug 26 12:03:00 2022] GET /identity/v3/users/d8c4341f54a041e79f3c3e65f708dbed => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:03:00.605994 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2087bfc6-77f0-4e46-bf81-1131c18399f0 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:03:00.665337 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:03:00.702301 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2087bfc6-77f0-4e46-bf81-1131c18399f0 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:03:00.734481 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3930e1ef-afa0-4ac2-a770-601103cff928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:03:00.735173 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3930e1ef-afa0-4ac2-a770-601103cff928 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:03:00.735489 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3930e1ef-afa0-4ac2-a770-601103cff928 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:03:00.735759 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3930e1ef-afa0-4ac2-a770-601103cff928 None admin] PATH_INFO: `/v3/projects/06d12a8c45ae46709be68115b9e058f9` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:03:00.736849 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3930e1ef-afa0-4ac2-a770-601103cff928 None admin] RBAC: Authorizing `identity:get_project(project_id=06d12a8c45ae46709be68115b9e058f9)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:03:00.738262 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3930e1ef-afa0-4ac2-a770-601103cff928 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:03:00.740251 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 572/1152] 10.222.0.61 () {56 vars in 1186 bytes} [Fri Aug 26 12:03:00 2022] GET /identity/v3/projects/06d12a8c45ae46709be68115b9e058f9 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:03:00.746391 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70aec62-b29b-48fa-83fb-0a686dea50d9 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:03:00.791885 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:03:00.844330 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70aec62-b29b-48fa-83fb-0a686dea50d9 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:03:00.875642 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94287ce9-74e2-48d3-b9d5-5892a7d858a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:03:00.876273 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94287ce9-74e2-48d3-b9d5-5892a7d858a9 None admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:03:00.876553 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94287ce9-74e2-48d3-b9d5-5892a7d858a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:03:00.876821 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94287ce9-74e2-48d3-b9d5-5892a7d858a9 None admin] PATH_INFO: `/v3/projects/06d12a8c45ae46709be68115b9e058f9/users/d8c4341f54a041e79f3c3e65f708dbed/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:03:00.878656 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94287ce9-74e2-48d3-b9d5-5892a7d858a9 None admin] RBAC: Authorizing `identity:create_grant(project_id=06d12a8c45ae46709be68115b9e058f9, user_id=d8c4341f54a041e79f3c3e65f708dbed, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:03:00.880095 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94287ce9-74e2-48d3-b9d5-5892a7d858a9 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:03:00.893895 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 581/1153] 10.222.0.61 () {58 vars in 1439 bytes} [Fri Aug 26 12:03:00 2022] PUT /identity/v3/projects/06d12a8c45ae46709be68115b9e058f9/users/d8c4341f54a041e79f3c3e65f708dbed/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:03:02.776693 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fafec4b-942c-444c-b8fa-32015f3ceea7 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:03:02.777132 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fafec4b-942c-444c-b8fa-32015f3ceea7 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:03:02.777475 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fafec4b-942c-444c-b8fa-32015f3ceea7 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:03:02.778813 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 573/1154] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:03:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:03:02.783874 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5da2ec-7710-43ea-8b36-a61596cdf196 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:03:02.784293 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5da2ec-7710-43ea-8b36-a61596cdf196 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:03:02.784429 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5da2ec-7710-43ea-8b36-a61596cdf196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:03:02.801866 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-7d5da2ec-7710-43ea-8b36-a61596cdf196 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:03:02.803032 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-7d5da2ec-7710-43ea-8b36-a61596cdf196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:03:02.868818 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 582/1155] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:03:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 12:03:03.142895 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fafec4b-942c-444c-b8fa-32015f3ceea7 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:03:03.241685 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fafec4b-942c-444c-b8fa-32015f3ceea7 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:03:03.269696 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:03:03.274612 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21a4d67-06de-44eb-90e3-0ea3a1c9fd51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:03:03.275236 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a4d67-06de-44eb-90e3-0ea3a1c9fd51 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:03:03.275521 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a4d67-06de-44eb-90e3-0ea3a1c9fd51 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:03:03.275791 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a4d67-06de-44eb-90e3-0ea3a1c9fd51 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:03:03.285960 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21a4d67-06de-44eb-90e3-0ea3a1c9fd51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:03:03.290557 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-b21a4d67-06de-44eb-90e3-0ea3a1c9fd51 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 12:03:03.290557 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:03:03.290557 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:03:03.290557 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:03:03.290557 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:03:03.290557 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:03:03.290557 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:03:03.290557 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:03:03.290557 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:03:03.290557 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:03:03.290557 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:03:03.290557 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:03:03.290557 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:03:03.290557 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:03:03.290557 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:03:03.290557 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:03:03.291679 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:03:03.291679 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:03:03.291679 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:03:03.291679 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:03:03.291679 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:03:03.291679 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:03:03.291679 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:03:03.291679 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:03:03.291679 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:03:03.291679 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:03:03.291679 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:03:03.291679 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:03:03.291679 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:03:03.291679 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:03:03.291679 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:03:03.291679 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:03:03.293421 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:03:03.293421 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:03:03.293421 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:03:03.293421 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:03:03.293421 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:03:03.293421 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:03:03.293421 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:03:03.293421 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:03:03.293421 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:03:03.293421 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:03:03.293421 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 12:03:03.293421 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:03:03.293421 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 574/1156] 10.222.0.61 () {56 vars in 1096 bytes} [Fri Aug 26 12:03:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:03:03.298166 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5da2ec-7710-43ea-8b36-a61596cdf196 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:03:03.332307 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:03:03.397475 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5da2ec-7710-43ea-8b36-a61596cdf196 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:03:03.430415 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63abb41-ba29-4588-80bc-f75d51a76cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:03:03.431009 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63abb41-ba29-4588-80bc-f75d51a76cd8 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:03:03.431321 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63abb41-ba29-4588-80bc-f75d51a76cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:03:03.431594 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63abb41-ba29-4588-80bc-f75d51a76cd8 None admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:03:03.432249 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63abb41-ba29-4588-80bc-f75d51a76cd8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:03:03.432553 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63abb41-ba29-4588-80bc-f75d51a76cd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:03:03.434500 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63abb41-ba29-4588-80bc-f75d51a76cd8 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:03:03.442445 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 583/1157] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:03:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:03:03.448045 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21a4d67-06de-44eb-90e3-0ea3a1c9fd51 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:03:03.545641 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21a4d67-06de-44eb-90e3-0ea3a1c9fd51 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:03:03.577610 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4332066c-4eb9-4415-b089-3822e333742d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:03:03.578216 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4332066c-4eb9-4415-b089-3822e333742d None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:03:03.578491 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4332066c-4eb9-4415-b089-3822e333742d None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:03:03.578759 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4332066c-4eb9-4415-b089-3822e333742d None admin] PATH_INFO: `/v3/users/d8c4341f54a041e79f3c3e65f708dbed` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:03:03.579932 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4332066c-4eb9-4415-b089-3822e333742d None admin] RBAC: Authorizing `identity:get_user(user_id=d8c4341f54a041e79f3c3e65f708dbed)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:03:03.581344 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4332066c-4eb9-4415-b089-3822e333742d None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:03:03.582882 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 575/1158] 10.222.0.61 () {56 vars in 1177 bytes} [Fri Aug 26 12:03:03 2022] GET /identity/v3/users/d8c4341f54a041e79f3c3e65f708dbed => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:03:03.588649 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63abb41-ba29-4588-80bc-f75d51a76cd8 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:03:03.682627 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:03:03.686493 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63abb41-ba29-4588-80bc-f75d51a76cd8 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:03:03.718806 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9bf4af-6563-41f5-b7d4-2815274e0779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:03:03.719417 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9bf4af-6563-41f5-b7d4-2815274e0779 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:03:03.719692 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9bf4af-6563-41f5-b7d4-2815274e0779 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:03:03.719958 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9bf4af-6563-41f5-b7d4-2815274e0779 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:03:03.784774 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9bf4af-6563-41f5-b7d4-2815274e0779 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:03:03.786206 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9bf4af-6563-41f5-b7d4-2815274e0779 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:03:03.876111 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:03:03.880617 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 584/1159] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:03:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 12:03:03.887042 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4332066c-4eb9-4415-b089-3822e333742d None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:03:03.944231 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:03:03.983823 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4332066c-4eb9-4415-b089-3822e333742d None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:03:04.015668 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7c57c4-1e7b-4171-bbce-31f615130732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:03:04.016261 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7c57c4-1e7b-4171-bbce-31f615130732 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:03:04.016540 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7c57c4-1e7b-4171-bbce-31f615130732 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:03:04.016806 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7c57c4-1e7b-4171-bbce-31f615130732 None admin] PATH_INFO: `/v3/projects/06d12a8c45ae46709be68115b9e058f9` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:03:04.017892 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7c57c4-1e7b-4171-bbce-31f615130732 None admin] RBAC: Authorizing `identity:get_project(project_id=06d12a8c45ae46709be68115b9e058f9)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:03:04.019324 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7c57c4-1e7b-4171-bbce-31f615130732 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:03:04.020896 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 576/1160] 10.222.0.61 () {56 vars in 1186 bytes} [Fri Aug 26 12:03:03 2022] GET /identity/v3/projects/06d12a8c45ae46709be68115b9e058f9 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:03:04.026863 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9bf4af-6563-41f5-b7d4-2815274e0779 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:03:04.121909 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9bf4af-6563-41f5-b7d4-2815274e0779 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:03:04.154033 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19da01be-b179-4a88-9c9c-91ef5b471f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:03:04.154610 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da01be-b179-4a88-9c9c-91ef5b471f64 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:03:04.154882 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da01be-b179-4a88-9c9c-91ef5b471f64 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:03:04.155148 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da01be-b179-4a88-9c9c-91ef5b471f64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:03:04.155882 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19da01be-b179-4a88-9c9c-91ef5b471f64 None admin] RBAC: Adding query filter params (role.id=70ec1ba484714c8ca3adb8fe5845107c, scope.project.id=06d12a8c45ae46709be68115b9e058f9, user.id=d8c4341f54a041e79f3c3e65f708dbed) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:03:04.156180 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19da01be-b179-4a88-9c9c-91ef5b471f64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:03:04.157580 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19da01be-b179-4a88-9c9c-91ef5b471f64 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:03:04.163045 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 585/1161] 10.222.0.61 () {56 vars in 1374 bytes} [Fri Aug 26 12:03:04 2022] GET /identity/v3/role_assignments?user.id=d8c4341f54a041e79f3c3e65f708dbed&scope.project.id=06d12a8c45ae46709be68115b9e058f9&role.id=70ec1ba484714c8ca3adb8fe5845107c => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:03:06.062869 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47600d2-b5d3-4555-aca9-25ff8ee000ed None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:03:06.063393 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47600d2-b5d3-4555-aca9-25ff8ee000ed None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:03:06.063393 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47600d2-b5d3-4555-aca9-25ff8ee000ed None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:03:06.064701 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 577/1162] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:03:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:03:06.069885 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a9bb56-cb7c-4498-bfcb-53946b598f1f None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:03:06.070254 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a9bb56-cb7c-4498-bfcb-53946b598f1f None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:03:06.070424 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a9bb56-cb7c-4498-bfcb-53946b598f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:03:06.088709 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-f4a9bb56-cb7c-4498-bfcb-53946b598f1f None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:03:06.090616 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-f4a9bb56-cb7c-4498-bfcb-53946b598f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:03:06.144701 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:03:06.155980 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 586/1163] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:03:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 12:03:06.431044 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47600d2-b5d3-4555-aca9-25ff8ee000ed None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:03:06.530767 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47600d2-b5d3-4555-aca9-25ff8ee000ed None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:03:06.555334 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:03:06.564141 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03bbf74-9bae-4022-8bae-2d2f8163035c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:03:06.564739 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03bbf74-9bae-4022-8bae-2d2f8163035c None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:03:06.565021 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03bbf74-9bae-4022-8bae-2d2f8163035c None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:03:06.565293 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03bbf74-9bae-4022-8bae-2d2f8163035c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:03:06.630509 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03bbf74-9bae-4022-8bae-2d2f8163035c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:03:06.631974 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03bbf74-9bae-4022-8bae-2d2f8163035c None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:03:06.725898 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 578/1164] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:03:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 12:03:06.732306 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a9bb56-cb7c-4498-bfcb-53946b598f1f None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:03:06.828666 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a9bb56-cb7c-4498-bfcb-53946b598f1f None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:03:06.861281 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd95b2e-891f-428f-a50c-4fb186e9b4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:03:06.861897 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd95b2e-891f-428f-a50c-4fb186e9b4d2 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:03:06.862173 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd95b2e-891f-428f-a50c-4fb186e9b4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:03:06.862442 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd95b2e-891f-428f-a50c-4fb186e9b4d2 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:03:06.868895 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd95b2e-891f-428f-a50c-4fb186e9b4d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:03:06.870316 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd95b2e-891f-428f-a50c-4fb186e9b4d2 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:03:06.881349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-7cd95b2e-891f-428f-a50c-4fb186e9b4d2 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 26 12:03:06.881349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:03:06.881349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 26 12:03:06.881349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 26 12:03:06.881349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 12:03:06.881349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 12:03:06.881349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 12:03:06.881349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 12:03:06.881349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 26 12:03:06.881349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:03:06.881349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 12:03:06.881349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:03:06.881349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:03:06.881349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:03:06.881349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:03:06.881349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:03:06.881349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:03:06.882566 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:03:06.882566 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:03:06.882566 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:03:06.882566 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:03:06.882566 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:03:06.882566 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:03:06.882566 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:03:06.882566 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:03:06.882566 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:03:06.882566 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:03:06.882566 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:03:06.882566 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:03:06.882566 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:03:06.882566 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:03:06.882566 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:03:06.882566 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:03:06.883910 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:03:06.883910 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:03:06.883910 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:03:06.883910 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:03:06.883910 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:03:06.883910 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:03:06.883910 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:03:06.883910 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:03:06.883910 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:03:06.883910 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:03:06.883910 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 26 12:03:06.883910 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:03:06.883910 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 26 12:03:06.883910 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:03:06.883910 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 587/1165] 10.222.0.61 () {56 vars in 1117 bytes} [Fri Aug 26 12:03:06 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:03:06.889556 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03bbf74-9bae-4022-8bae-2d2f8163035c None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:03:06.916149 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:03:06.985053 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03bbf74-9bae-4022-8bae-2d2f8163035c None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:03:07.017557 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caf0e33-1703-40ca-872a-ea92f029a81b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:03:07.018149 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf0e33-1703-40ca-872a-ea92f029a81b None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:03:07.018426 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf0e33-1703-40ca-872a-ea92f029a81b None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:03:07.018698 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf0e33-1703-40ca-872a-ea92f029a81b None admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:03:07.019352 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caf0e33-1703-40ca-872a-ea92f029a81b None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:03:07.019653 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caf0e33-1703-40ca-872a-ea92f029a81b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:03:07.021004 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caf0e33-1703-40ca-872a-ea92f029a81b None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:03:07.028395 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 579/1166] 10.222.0.61 () {56 vars in 1115 bytes} [Fri Aug 26 12:03:06 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:03:08.911600 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de53d33-6bc7-48b5-b7ad-78ca2cbce15b None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:03:08.912110 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de53d33-6bc7-48b5-b7ad-78ca2cbce15b None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:03:08.912223 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de53d33-6bc7-48b5-b7ad-78ca2cbce15b None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:03:08.913481 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 588/1167] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:03:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:03:08.919122 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8ad800-1b0f-40c5-92c1-6b65825aca14 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:03:08.919548 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8ad800-1b0f-40c5-92c1-6b65825aca14 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:03:08.919698 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8ad800-1b0f-40c5-92c1-6b65825aca14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:03:08.937088 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-2a8ad800-1b0f-40c5-92c1-6b65825aca14 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:03:08.938260 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-2a8ad800-1b0f-40c5-92c1-6b65825aca14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:03:09.003615 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 580/1168] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:03:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 12:03:09.269063 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de53d33-6bc7-48b5-b7ad-78ca2cbce15b None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:03:09.321053 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:03:09.367544 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de53d33-6bc7-48b5-b7ad-78ca2cbce15b None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:03:09.400160 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d252d04-d031-451e-a9be-757563be2cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:03:09.400749 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d252d04-d031-451e-a9be-757563be2cd9 None admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:03:09.401022 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d252d04-d031-451e-a9be-757563be2cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:03:09.401287 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d252d04-d031-451e-a9be-757563be2cd9 None admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:03:09.402520 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d252d04-d031-451e-a9be-757563be2cd9 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:03:09.403894 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d252d04-d031-451e-a9be-757563be2cd9 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:03:09.418501 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 589/1169] 10.222.0.61 () {60 vars in 1138 bytes} [Fri Aug 26 12:03:09 2022] POST /identity/v3/domains => generated 267 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:03:11.326763 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d44338-3e0a-48a0-a809-c669d87cda12 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:03:11.327336 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d44338-3e0a-48a0-a809-c669d87cda12 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:03:11.327490 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d44338-3e0a-48a0-a809-c669d87cda12 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:03:11.328865 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 581/1170] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:03:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:03:11.334124 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135e87aa-ff01-4dca-9958-abb2aebd1c89 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:03:11.334490 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135e87aa-ff01-4dca-9958-abb2aebd1c89 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:03:11.334689 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135e87aa-ff01-4dca-9958-abb2aebd1c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:03:11.353121 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-135e87aa-ff01-4dca-9958-abb2aebd1c89 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:03:11.354326 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-135e87aa-ff01-4dca-9958-abb2aebd1c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:03:11.419432 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 590/1171] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:03:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 12:03:11.685514 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d44338-3e0a-48a0-a809-c669d87cda12 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:03:11.709070 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:03:11.786146 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d44338-3e0a-48a0-a809-c669d87cda12 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:03:11.818665 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb49eb22-64db-4a64-bf40-d08fc0fc28cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:03:11.819311 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb49eb22-64db-4a64-bf40-d08fc0fc28cf None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:03:11.819593 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb49eb22-64db-4a64-bf40-d08fc0fc28cf None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:03:11.819865 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb49eb22-64db-4a64-bf40-d08fc0fc28cf None admin] PATH_INFO: `/v3/domains/1dae31a896a54431be92f733bd56281b` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:03:11.829842 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb49eb22-64db-4a64-bf40-d08fc0fc28cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=1dae31a896a54431be92f733bd56281b)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:03:11.831296 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb49eb22-64db-4a64-bf40-d08fc0fc28cf None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:03:11.832852 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 582/1172] 10.222.0.61 () {56 vars in 1183 bytes} [Fri Aug 26 12:03:11 2022] GET /identity/v3/domains/1dae31a896a54431be92f733bd56281b => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:03:11.838835 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135e87aa-ff01-4dca-9958-abb2aebd1c89 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:03:11.881165 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:03:11.937031 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135e87aa-ff01-4dca-9958-abb2aebd1c89 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:03:11.969336 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a0859d-cb68-42d6-8cc2-8e1e411b6d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:03:11.969929 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a0859d-cb68-42d6-8cc2-8e1e411b6d87 None admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:03:11.970208 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a0859d-cb68-42d6-8cc2-8e1e411b6d87 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:03:11.970483 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a0859d-cb68-42d6-8cc2-8e1e411b6d87 None admin] PATH_INFO: `/v3/projects` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:03:11.971124 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a0859d-cb68-42d6-8cc2-8e1e411b6d87 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:03:11.972560 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a0859d-cb68-42d6-8cc2-8e1e411b6d87 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:03:11.998518 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 591/1173] 10.222.0.61 () {60 vars in 1141 bytes} [Fri Aug 26 12:03:11 2022] POST /identity/v3/projects => generated 361 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:03:13.913951 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03149db0-d820-44df-b56b-668ae69a1308 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:03:13.914487 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03149db0-d820-44df-b56b-668ae69a1308 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:03:13.914487 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03149db0-d820-44df-b56b-668ae69a1308 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:03:13.915854 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 583/1174] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:03:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:03:13.921526 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021acc7a-4f5b-4bb4-9296-0d7c83f62f8b None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:03:13.921917 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021acc7a-4f5b-4bb4-9296-0d7c83f62f8b None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:03:13.922076 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021acc7a-4f5b-4bb4-9296-0d7c83f62f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:03:13.940144 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-021acc7a-4f5b-4bb4-9296-0d7c83f62f8b None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:03:13.941311 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-021acc7a-4f5b-4bb4-9296-0d7c83f62f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:03:14.004791 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 592/1175] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:03:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 12:03:14.272141 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03149db0-d820-44df-b56b-668ae69a1308 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:03:14.353993 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:03:14.372845 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03149db0-d820-44df-b56b-668ae69a1308 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:03:14.405835 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8489c2a-b84a-4103-965a-deee07a667e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:03:14.406453 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8489c2a-b84a-4103-965a-deee07a667e3 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:03:14.406732 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8489c2a-b84a-4103-965a-deee07a667e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:03:14.407002 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8489c2a-b84a-4103-965a-deee07a667e3 None admin] PATH_INFO: `/v3/domains/1dae31a896a54431be92f733bd56281b` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:03:14.408136 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8489c2a-b84a-4103-965a-deee07a667e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=1dae31a896a54431be92f733bd56281b)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:03:14.409568 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8489c2a-b84a-4103-965a-deee07a667e3 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:03:14.411112 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 584/1176] 10.222.0.61 () {56 vars in 1183 bytes} [Fri Aug 26 12:03:14 2022] GET /identity/v3/domains/1dae31a896a54431be92f733bd56281b => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:03:14.416761 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021acc7a-4f5b-4bb4-9296-0d7c83f62f8b None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:03:14.440256 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:03:14.513576 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021acc7a-4f5b-4bb4-9296-0d7c83f62f8b None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:03:14.546370 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d17a9ab-7939-4a78-8ed8-df7235cde660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:03:14.546898 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d17a9ab-7939-4a78-8ed8-df7235cde660 None admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:03:14.547172 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d17a9ab-7939-4a78-8ed8-df7235cde660 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:03:14.547469 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d17a9ab-7939-4a78-8ed8-df7235cde660 None admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:03:14.548100 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d17a9ab-7939-4a78-8ed8-df7235cde660 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:03:14.549512 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d17a9ab-7939-4a78-8ed8-df7235cde660 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:03:14.571014 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 593/1177] 10.222.0.61 () {60 vars in 1132 bytes} [Fri Aug 26 12:03:14 2022] POST /identity/v3/users => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:03:16.495225 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c1b105-c01c-4a5d-91ee-46d8e4f3f1b2 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:03:16.495809 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c1b105-c01c-4a5d-91ee-46d8e4f3f1b2 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:03:16.495940 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c1b105-c01c-4a5d-91ee-46d8e4f3f1b2 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:03:16.497401 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 585/1178] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:03:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:03:16.502474 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229af7f4-ef63-493b-ac76-a94d9f61ebbf None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:03:16.502898 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229af7f4-ef63-493b-ac76-a94d9f61ebbf None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:03:16.503106 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229af7f4-ef63-493b-ac76-a94d9f61ebbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:03:16.520870 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-229af7f4-ef63-493b-ac76-a94d9f61ebbf None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:03:16.522033 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-229af7f4-ef63-493b-ac76-a94d9f61ebbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:03:16.586540 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 594/1179] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:03:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 12:03:16.851545 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c1b105-c01c-4a5d-91ee-46d8e4f3f1b2 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:03:16.950792 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c1b105-c01c-4a5d-91ee-46d8e4f3f1b2 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:03:16.982437 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae35b4c8-49c6-4327-a40d-27723660a947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:03:16.983048 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae35b4c8-49c6-4327-a40d-27723660a947 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:03:16.983362 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae35b4c8-49c6-4327-a40d-27723660a947 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:03:16.983634 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae35b4c8-49c6-4327-a40d-27723660a947 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:03:16.993686 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae35b4c8-49c6-4327-a40d-27723660a947 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:03:16.998892 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-ae35b4c8-49c6-4327-a40d-27723660a947 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 12:03:16.998892 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:03:16.998892 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:03:16.998892 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:03:16.998892 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:03:16.998892 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:03:16.998892 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:03:16.998892 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:03:16.998892 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:03:16.998892 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:03:16.998892 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:03:16.998892 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:03:16.998892 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:03:16.998892 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:03:16.998892 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:03:16.998892 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:03:17.000092 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:03:17.000092 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:03:17.000092 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:03:17.000092 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:03:17.000092 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:03:17.000092 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:03:17.000092 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:03:17.000092 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:03:17.000092 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:03:17.000092 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:03:17.000092 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:03:17.000092 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:03:17.000092 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:03:17.000092 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:03:17.000092 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:03:17.000092 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:03:17.001205 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:03:17.001205 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:03:17.001205 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:03:17.001205 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:03:17.001205 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:03:17.001205 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:03:17.001205 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:03:17.001205 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:03:17.001205 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:03:17.001205 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:03:17.001205 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 12:03:17.001205 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:03:17.001205 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 586/1180] 10.222.0.61 () {56 vars in 1096 bytes} [Fri Aug 26 12:03:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:03:17.005944 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229af7f4-ef63-493b-ac76-a94d9f61ebbf None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:03:17.018114 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:03:17.102777 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229af7f4-ef63-493b-ac76-a94d9f61ebbf None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:03:17.135496 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51daaa0c-54a3-463e-a7ec-7c8c399907e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:03:17.136138 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51daaa0c-54a3-463e-a7ec-7c8c399907e9 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:03:17.136943 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51daaa0c-54a3-463e-a7ec-7c8c399907e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:03:17.137213 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51daaa0c-54a3-463e-a7ec-7c8c399907e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:03:17.137864 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51daaa0c-54a3-463e-a7ec-7c8c399907e9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:03:17.138168 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51daaa0c-54a3-463e-a7ec-7c8c399907e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:03:17.139707 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51daaa0c-54a3-463e-a7ec-7c8c399907e9 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:03:17.148929 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 595/1181] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:03:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:03:17.155033 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae35b4c8-49c6-4327-a40d-27723660a947 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:03:17.167450 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:03:17.253556 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae35b4c8-49c6-4327-a40d-27723660a947 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:03:17.286593 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd32fed-78a6-46ca-8ae5-2a26af4cec5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:03:17.287230 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd32fed-78a6-46ca-8ae5-2a26af4cec5b None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:03:17.287512 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd32fed-78a6-46ca-8ae5-2a26af4cec5b None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:03:17.287786 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd32fed-78a6-46ca-8ae5-2a26af4cec5b None admin] PATH_INFO: `/v3/users/aa234db615d248e49b3506a60ea79cff` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:03:17.300909 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd32fed-78a6-46ca-8ae5-2a26af4cec5b None admin] RBAC: Authorizing `identity:get_user(user_id=aa234db615d248e49b3506a60ea79cff)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:03:17.302348 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd32fed-78a6-46ca-8ae5-2a26af4cec5b None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:03:17.303943 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 587/1182] 10.222.0.61 () {56 vars in 1177 bytes} [Fri Aug 26 12:03:17 2022] GET /identity/v3/users/aa234db615d248e49b3506a60ea79cff => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:03:17.310616 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51daaa0c-54a3-463e-a7ec-7c8c399907e9 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:03:17.380257 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:03:17.410521 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51daaa0c-54a3-463e-a7ec-7c8c399907e9 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:03:17.441869 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3753de51-465f-440a-b5d6-f73cc1b072cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:03:17.442452 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3753de51-465f-440a-b5d6-f73cc1b072cf None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:03:17.442726 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3753de51-465f-440a-b5d6-f73cc1b072cf None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:03:17.442991 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3753de51-465f-440a-b5d6-f73cc1b072cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:03:17.506778 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3753de51-465f-440a-b5d6-f73cc1b072cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:03:17.508495 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3753de51-465f-440a-b5d6-f73cc1b072cf None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:03:17.575130 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:03:17.607860 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 596/1183] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:03:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 12:03:17.613770 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd32fed-78a6-46ca-8ae5-2a26af4cec5b None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:03:17.680042 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:03:17.715992 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd32fed-78a6-46ca-8ae5-2a26af4cec5b None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:03:17.750935 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079aa915-5bb7-4263-86dd-faf6c916c993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:03:17.751601 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079aa915-5bb7-4263-86dd-faf6c916c993 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:03:17.751886 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079aa915-5bb7-4263-86dd-faf6c916c993 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:03:17.752161 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079aa915-5bb7-4263-86dd-faf6c916c993 None admin] PATH_INFO: `/v3/projects/f40e201b5d404eca82c18dfd22ccb5e1` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:03:17.753290 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079aa915-5bb7-4263-86dd-faf6c916c993 None admin] RBAC: Authorizing `identity:get_project(project_id=f40e201b5d404eca82c18dfd22ccb5e1)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:03:17.754715 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079aa915-5bb7-4263-86dd-faf6c916c993 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:03:17.756386 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 588/1184] 10.222.0.61 () {56 vars in 1186 bytes} [Fri Aug 26 12:03:17 2022] GET /identity/v3/projects/f40e201b5d404eca82c18dfd22ccb5e1 => generated 361 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:03:17.763317 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3753de51-465f-440a-b5d6-f73cc1b072cf None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:03:17.858645 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3753de51-465f-440a-b5d6-f73cc1b072cf None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:03:17.892276 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b2113c-5876-48e8-97ca-c8225e7dc2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:03:17.892872 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b2113c-5876-48e8-97ca-c8225e7dc2bc None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:03:17.893152 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b2113c-5876-48e8-97ca-c8225e7dc2bc None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:03:17.893424 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b2113c-5876-48e8-97ca-c8225e7dc2bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:03:17.894143 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b2113c-5876-48e8-97ca-c8225e7dc2bc None admin] RBAC: Adding query filter params (role.id=70ec1ba484714c8ca3adb8fe5845107c, scope.project.id=f40e201b5d404eca82c18dfd22ccb5e1, user.id=aa234db615d248e49b3506a60ea79cff) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:03:17.894448 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b2113c-5876-48e8-97ca-c8225e7dc2bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:03:17.895891 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b2113c-5876-48e8-97ca-c8225e7dc2bc None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:03:17.901150 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 597/1185] 10.222.0.61 () {56 vars in 1374 bytes} [Fri Aug 26 12:03:17 2022] GET /identity/v3/role_assignments?user.id=aa234db615d248e49b3506a60ea79cff&scope.project.id=f40e201b5d404eca82c18dfd22ccb5e1&role.id=70ec1ba484714c8ca3adb8fe5845107c => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:03:19.770262 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb36ed3-d4ab-47d2-98e6-8d139ae4b044 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:03:19.770729 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb36ed3-d4ab-47d2-98e6-8d139ae4b044 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:03:19.770821 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb36ed3-d4ab-47d2-98e6-8d139ae4b044 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:03:19.772180 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 589/1186] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:03:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:03:19.777363 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d14f1ed-19b3-41cf-8067-4ccadb99dd2b None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:03:19.777781 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d14f1ed-19b3-41cf-8067-4ccadb99dd2b None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:03:19.777892 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d14f1ed-19b3-41cf-8067-4ccadb99dd2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:03:19.795847 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-1d14f1ed-19b3-41cf-8067-4ccadb99dd2b None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:03:19.797003 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-1d14f1ed-19b3-41cf-8067-4ccadb99dd2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:03:19.825145 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:03:19.862837 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 598/1187] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:03:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 12:03:20.132793 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb36ed3-d4ab-47d2-98e6-8d139ae4b044 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:03:20.233298 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb36ed3-d4ab-47d2-98e6-8d139ae4b044 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:03:20.266662 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc0cf70-b42a-4ce8-bd82-b6f798df56f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:03:20.267309 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc0cf70-b42a-4ce8-bd82-b6f798df56f2 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:03:20.267632 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc0cf70-b42a-4ce8-bd82-b6f798df56f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:03:20.267907 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc0cf70-b42a-4ce8-bd82-b6f798df56f2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:03:20.270487 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:03:20.279951 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc0cf70-b42a-4ce8-bd82-b6f798df56f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:03:20.284591 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-0fc0cf70-b42a-4ce8-bd82-b6f798df56f2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 12:03:20.284591 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:03:20.284591 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:03:20.284591 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:03:20.284591 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:03:20.284591 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:03:20.284591 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:03:20.284591 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:03:20.284591 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:03:20.284591 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:03:20.284591 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:03:20.284591 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:03:20.284591 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:03:20.284591 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:03:20.284591 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:03:20.284591 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:03:20.285696 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:03:20.285696 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:03:20.285696 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:03:20.285696 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:03:20.285696 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:03:20.285696 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:03:20.285696 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:03:20.285696 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:03:20.285696 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:03:20.285696 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:03:20.285696 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:03:20.285696 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:03:20.285696 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:03:20.285696 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:03:20.285696 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:03:20.285696 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:03:20.286721 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:03:20.286721 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:03:20.286721 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:03:20.286721 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:03:20.286721 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:03:20.286721 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:03:20.286721 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:03:20.286721 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:03:20.286721 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:03:20.286721 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:03:20.286721 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 12:03:20.286721 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:03:20.286721 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 590/1188] 10.222.0.61 () {56 vars in 1096 bytes} [Fri Aug 26 12:03:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:03:20.291745 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d14f1ed-19b3-41cf-8067-4ccadb99dd2b None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:03:20.389734 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d14f1ed-19b3-41cf-8067-4ccadb99dd2b None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:03:20.422221 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e578e9-4ee1-45b3-9ea6-5a37f7001d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:03:20.422814 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e578e9-4ee1-45b3-9ea6-5a37f7001d96 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:03:20.423093 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e578e9-4ee1-45b3-9ea6-5a37f7001d96 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:03:20.423395 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e578e9-4ee1-45b3-9ea6-5a37f7001d96 None admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:03:20.424048 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e578e9-4ee1-45b3-9ea6-5a37f7001d96 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:03:20.424349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e578e9-4ee1-45b3-9ea6-5a37f7001d96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:03:20.425704 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e578e9-4ee1-45b3-9ea6-5a37f7001d96 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:03:20.433458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 599/1189] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:03:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:03:20.439182 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc0cf70-b42a-4ce8-bd82-b6f798df56f2 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:03:20.535838 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc0cf70-b42a-4ce8-bd82-b6f798df56f2 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:03:20.567516 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e7f82-9859-4dde-8dd8-e41d87f5d91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:03:20.568120 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e7f82-9859-4dde-8dd8-e41d87f5d91d None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:03:20.568399 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e7f82-9859-4dde-8dd8-e41d87f5d91d None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:03:20.568665 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e7f82-9859-4dde-8dd8-e41d87f5d91d None admin] PATH_INFO: `/v3/users/aa234db615d248e49b3506a60ea79cff` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:03:20.569756 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5e7f82-9859-4dde-8dd8-e41d87f5d91d None admin] RBAC: Authorizing `identity:get_user(user_id=aa234db615d248e49b3506a60ea79cff)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:03:20.571174 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5e7f82-9859-4dde-8dd8-e41d87f5d91d None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:03:20.572758 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 591/1190] 10.222.0.61 () {56 vars in 1177 bytes} [Fri Aug 26 12:03:20 2022] GET /identity/v3/users/aa234db615d248e49b3506a60ea79cff => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:03:20.579009 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e578e9-4ee1-45b3-9ea6-5a37f7001d96 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:03:20.607192 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:03:20.677880 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e578e9-4ee1-45b3-9ea6-5a37f7001d96 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:03:20.710536 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db6d56a-ab13-4e50-b287-c484cdf98cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:03:20.711142 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db6d56a-ab13-4e50-b287-c484cdf98cbe None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:03:20.711449 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db6d56a-ab13-4e50-b287-c484cdf98cbe None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:03:20.711722 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db6d56a-ab13-4e50-b287-c484cdf98cbe None admin] PATH_INFO: `/v3/projects/f40e201b5d404eca82c18dfd22ccb5e1` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:03:20.712835 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db6d56a-ab13-4e50-b287-c484cdf98cbe None admin] RBAC: Authorizing `identity:get_project(project_id=f40e201b5d404eca82c18dfd22ccb5e1)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:03:20.714381 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db6d56a-ab13-4e50-b287-c484cdf98cbe None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:03:20.716250 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 600/1191] 10.222.0.61 () {56 vars in 1186 bytes} [Fri Aug 26 12:03:20 2022] GET /identity/v3/projects/f40e201b5d404eca82c18dfd22ccb5e1 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:03:20.722138 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e7f82-9859-4dde-8dd8-e41d87f5d91d None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:03:20.783431 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:03:20.820004 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e7f82-9859-4dde-8dd8-e41d87f5d91d None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:03:20.852606 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9255a037-10a3-461d-acc9-c4275e08e259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:03:20.853249 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9255a037-10a3-461d-acc9-c4275e08e259 None admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:03:20.853527 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9255a037-10a3-461d-acc9-c4275e08e259 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:03:20.853801 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9255a037-10a3-461d-acc9-c4275e08e259 None admin] PATH_INFO: `/v3/projects/f40e201b5d404eca82c18dfd22ccb5e1/users/aa234db615d248e49b3506a60ea79cff/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:03:20.855776 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9255a037-10a3-461d-acc9-c4275e08e259 None admin] RBAC: Authorizing `identity:create_grant(project_id=f40e201b5d404eca82c18dfd22ccb5e1, user_id=aa234db615d248e49b3506a60ea79cff, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:03:20.857200 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9255a037-10a3-461d-acc9-c4275e08e259 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:03:20.869058 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 592/1192] 10.222.0.61 () {58 vars in 1439 bytes} [Fri Aug 26 12:03:20 2022] PUT /identity/v3/projects/f40e201b5d404eca82c18dfd22ccb5e1/users/aa234db615d248e49b3506a60ea79cff/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:03:22.757450 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65d113a-0546-440c-ba5a-45b01246131c None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:03:22.757894 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65d113a-0546-440c-ba5a-45b01246131c None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:03:22.757998 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65d113a-0546-440c-ba5a-45b01246131c None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:03:22.759633 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 601/1193] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:03:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:03:22.765305 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb21fd-c5ce-45b0-a866-4f18efa57ece None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:03:22.765680 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb21fd-c5ce-45b0-a866-4f18efa57ece None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:03:22.765853 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb21fd-c5ce-45b0-a866-4f18efa57ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:03:22.784342 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-5adb21fd-c5ce-45b0-a866-4f18efa57ece None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:03:22.785516 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-5adb21fd-c5ce-45b0-a866-4f18efa57ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:03:22.849269 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 593/1194] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:03:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 12:03:23.115399 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65d113a-0546-440c-ba5a-45b01246131c None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:03:23.210883 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:03:23.218026 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65d113a-0546-440c-ba5a-45b01246131c None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:03:23.251571 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefeb922-3a54-477d-be40-426b227f99d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:03:23.252208 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefeb922-3a54-477d-be40-426b227f99d5 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:03:23.252489 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefeb922-3a54-477d-be40-426b227f99d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:03:23.252759 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefeb922-3a54-477d-be40-426b227f99d5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:03:23.263706 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefeb922-3a54-477d-be40-426b227f99d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:03:23.268346 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-aefeb922-3a54-477d-be40-426b227f99d5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 12:03:23.268346 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:03:23.268346 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:03:23.268346 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:03:23.268346 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:03:23.268346 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:03:23.268346 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:03:23.268346 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:03:23.268346 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:03:23.268346 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:03:23.268346 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:03:23.268346 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:03:23.268346 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:03:23.268346 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:03:23.268346 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:03:23.268346 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:03:23.269425 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:03:23.269425 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:03:23.269425 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:03:23.269425 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:03:23.269425 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:03:23.269425 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:03:23.269425 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:03:23.269425 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:03:23.269425 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:03:23.269425 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:03:23.269425 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:03:23.269425 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:03:23.269425 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:03:23.269425 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:03:23.269425 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:03:23.269425 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:03:23.270493 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:03:23.270493 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:03:23.270493 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:03:23.270493 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:03:23.270493 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:03:23.270493 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:03:23.270493 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:03:23.270493 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:03:23.270493 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:03:23.270493 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:03:23.270493 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 12:03:23.270493 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:03:23.270493 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 602/1195] 10.222.0.61 () {56 vars in 1096 bytes} [Fri Aug 26 12:03:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:03:23.275445 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adb21fd-c5ce-45b0-a866-4f18efa57ece None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:03:23.328795 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:03:23.372977 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adb21fd-c5ce-45b0-a866-4f18efa57ece None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:03:23.405454 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda2ab02-b6ed-4051-a441-c866d24770ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:03:23.406039 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2ab02-b6ed-4051-a441-c866d24770ef None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:03:23.406316 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2ab02-b6ed-4051-a441-c866d24770ef None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:03:23.406586 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2ab02-b6ed-4051-a441-c866d24770ef None admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:03:23.407263 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda2ab02-b6ed-4051-a441-c866d24770ef None admin] RBAC: Adding query filter params (name=admin) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:03:23.407567 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda2ab02-b6ed-4051-a441-c866d24770ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:03:23.408916 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda2ab02-b6ed-4051-a441-c866d24770ef None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:03:23.416595 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 594/1196] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:03:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:03:23.422775 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefeb922-3a54-477d-be40-426b227f99d5 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:03:23.518849 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefeb922-3a54-477d-be40-426b227f99d5 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:03:23.551091 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e528646e-9774-42d8-a2c8-b6197aea9ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:03:23.551716 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528646e-9774-42d8-a2c8-b6197aea9ad0 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:03:23.551991 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528646e-9774-42d8-a2c8-b6197aea9ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:03:23.552263 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528646e-9774-42d8-a2c8-b6197aea9ad0 None admin] PATH_INFO: `/v3/users/aa234db615d248e49b3506a60ea79cff` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:03:23.553352 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e528646e-9774-42d8-a2c8-b6197aea9ad0 None admin] RBAC: Authorizing `identity:get_user(user_id=aa234db615d248e49b3506a60ea79cff)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:03:23.554759 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e528646e-9774-42d8-a2c8-b6197aea9ad0 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:03:23.556322 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 603/1197] 10.222.0.61 () {56 vars in 1177 bytes} [Fri Aug 26 12:03:23 2022] GET /identity/v3/users/aa234db615d248e49b3506a60ea79cff => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:03:23.562725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda2ab02-b6ed-4051-a441-c866d24770ef None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:03:23.660109 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda2ab02-b6ed-4051-a441-c866d24770ef None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:03:23.675881 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:03:23.693944 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2cf771-4b74-4b76-ba15-b1c878f5ea2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:03:23.694529 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2cf771-4b74-4b76-ba15-b1c878f5ea2a None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:03:23.694806 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2cf771-4b74-4b76-ba15-b1c878f5ea2a None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:03:23.695075 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2cf771-4b74-4b76-ba15-b1c878f5ea2a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:03:23.759446 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2cf771-4b74-4b76-ba15-b1c878f5ea2a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:03:23.760877 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2cf771-4b74-4b76-ba15-b1c878f5ea2a None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:03:23.854247 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 595/1198] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:03:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 12:03:23.860771 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e528646e-9774-42d8-a2c8-b6197aea9ad0 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:03:23.885364 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:03:23.958404 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e528646e-9774-42d8-a2c8-b6197aea9ad0 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:03:23.991706 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76801f4-e017-4d3b-a338-54f86bfc6d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:03:23.992315 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76801f4-e017-4d3b-a338-54f86bfc6d25 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:03:23.992596 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76801f4-e017-4d3b-a338-54f86bfc6d25 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:03:23.992867 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76801f4-e017-4d3b-a338-54f86bfc6d25 None admin] PATH_INFO: `/v3/projects/f40e201b5d404eca82c18dfd22ccb5e1` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:03:23.993966 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76801f4-e017-4d3b-a338-54f86bfc6d25 None admin] RBAC: Authorizing `identity:get_project(project_id=f40e201b5d404eca82c18dfd22ccb5e1)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:03:23.995899 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76801f4-e017-4d3b-a338-54f86bfc6d25 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:03:23.997512 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 604/1199] 10.222.0.61 () {56 vars in 1186 bytes} [Fri Aug 26 12:03:23 2022] GET /identity/v3/projects/f40e201b5d404eca82c18dfd22ccb5e1 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:03:24.003496 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2cf771-4b74-4b76-ba15-b1c878f5ea2a None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:03:24.020244 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:03:24.099809 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2cf771-4b74-4b76-ba15-b1c878f5ea2a None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:03:24.132060 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fa06f1-02b5-44d8-8497-883662835f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:03:24.132650 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa06f1-02b5-44d8-8497-883662835f09 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:03:24.132930 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa06f1-02b5-44d8-8497-883662835f09 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:03:24.133461 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa06f1-02b5-44d8-8497-883662835f09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:03:24.134174 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fa06f1-02b5-44d8-8497-883662835f09 None admin] RBAC: Adding query filter params (role.id=70ec1ba484714c8ca3adb8fe5845107c, scope.project.id=f40e201b5d404eca82c18dfd22ccb5e1, user.id=aa234db615d248e49b3506a60ea79cff) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:03:24.134481 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fa06f1-02b5-44d8-8497-883662835f09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:03:24.135935 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fa06f1-02b5-44d8-8497-883662835f09 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:03:24.142320 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 596/1200] 10.222.0.61 () {56 vars in 1374 bytes} [Fri Aug 26 12:03:24 2022] GET /identity/v3/role_assignments?user.id=aa234db615d248e49b3506a60ea79cff&scope.project.id=f40e201b5d404eca82c18dfd22ccb5e1&role.id=70ec1ba484714c8ca3adb8fe5845107c => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:03:33.987636 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=54 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:03:34.139746 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=60 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:19.562650 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e63d257-1b53-4740-835b-83ef97bbd173 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:19.563482 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e63d257-1b53-4740-835b-83ef97bbd173 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:19.563482 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e63d257-1b53-4740-835b-83ef97bbd173 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:19.564579 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 605/1201] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:04:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:04:19.570596 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442e4666-e1bb-4e5b-acbc-809cd16bd47e None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:19.571137 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442e4666-e1bb-4e5b-acbc-809cd16bd47e None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:19.571334 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442e4666-e1bb-4e5b-acbc-809cd16bd47e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:19.591074 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-442e4666-e1bb-4e5b-acbc-809cd16bd47e None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:04:19.592297 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-442e4666-e1bb-4e5b-acbc-809cd16bd47e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:04:19.674825 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 597/1202] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:04:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 12:04:19.957502 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e63d257-1b53-4740-835b-83ef97bbd173 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:20.085001 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e63d257-1b53-4740-835b-83ef97bbd173 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:20.128020 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a81489-0b6f-4978-b876-3816a88b4418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:20.128724 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a81489-0b6f-4978-b876-3816a88b4418 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:20.129021 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a81489-0b6f-4978-b876-3816a88b4418 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:20.129261 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a81489-0b6f-4978-b876-3816a88b4418 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:20.143240 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a81489-0b6f-4978-b876-3816a88b4418 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:20.144757 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a81489-0b6f-4978-b876-3816a88b4418 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:20.162808 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-89a81489-0b6f-4978-b876-3816a88b4418 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:04:20.162808 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:04:20.162808 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:04:20.162808 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:04:20.162808 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:04:20.162808 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:04:20.162808 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:04:20.162808 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:04:20.162808 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:04:20.162808 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:04:20.162808 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:04:20.162808 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:04:20.162808 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:04:20.162808 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:04:20.162808 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:04:20.162808 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:04:20.164242 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:04:20.164242 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:04:20.164242 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:04:20.164242 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:04:20.164242 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:04:20.164242 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:04:20.164242 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:04:20.164242 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:04:20.164242 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:04:20.164242 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:04:20.164242 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:04:20.164242 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:04:20.164242 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:04:20.164242 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:04:20.164242 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:04:20.164242 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:04:20.164242 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:04:20.165766 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:04:20.165766 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:04:20.165766 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:04:20.168866 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 606/1203] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:04:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:04:20.173606 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442e4666-e1bb-4e5b-acbc-809cd16bd47e None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:20.308802 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442e4666-e1bb-4e5b-acbc-809cd16bd47e None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:20.356462 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:20.364727 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5205ceb8-9292-460b-8611-5bceca0b6fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:20.365582 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205ceb8-9292-460b-8611-5bceca0b6fda None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:20.365952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205ceb8-9292-460b-8611-5bceca0b6fda None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:20.366229 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205ceb8-9292-460b-8611-5bceca0b6fda None admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:20.371270 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5205ceb8-9292-460b-8611-5bceca0b6fda None admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:04:20.371270 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5205ceb8-9292-460b-8611-5bceca0b6fda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:20.371270 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5205ceb8-9292-460b-8611-5bceca0b6fda None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:20.383429 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 598/1204] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:04:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:04:20.390376 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a81489-0b6f-4978-b876-3816a88b4418 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:20.464764 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:20.531586 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a81489-0b6f-4978-b876-3816a88b4418 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:20.577586 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39991f23-f4f6-40d1-a788-396a893c489d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:20.578221 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39991f23-f4f6-40d1-a788-396a893c489d None admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:20.578472 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39991f23-f4f6-40d1-a788-396a893c489d None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:20.578700 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39991f23-f4f6-40d1-a788-396a893c489d None admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:20.582533 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39991f23-f4f6-40d1-a788-396a893c489d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:20.584141 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39991f23-f4f6-40d1-a788-396a893c489d None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:20.612881 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 607/1205] 10.222.0.61 () {60 vars in 1132 bytes} [Fri Aug 26 12:04:20 2022] POST /identity/v3/users => generated 251 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:04:22.725205 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb29704-caad-4454-9650-bfed29ee8645 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:22.725793 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb29704-caad-4454-9650-bfed29ee8645 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:22.725793 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb29704-caad-4454-9650-bfed29ee8645 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:22.727152 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 599/1206] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:04:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:04:22.734138 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d081fb-389b-44d5-8d9f-4d4bb8eb9cd0 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:22.734683 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d081fb-389b-44d5-8d9f-4d4bb8eb9cd0 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:22.734683 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d081fb-389b-44d5-8d9f-4d4bb8eb9cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:22.759812 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-10d081fb-389b-44d5-8d9f-4d4bb8eb9cd0 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:04:22.761234 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-10d081fb-389b-44d5-8d9f-4d4bb8eb9cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:04:22.833790 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 608/1207] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:04:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 12:04:23.111516 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb29704-caad-4454-9650-bfed29ee8645 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:23.223028 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb29704-caad-4454-9650-bfed29ee8645 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:23.259864 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bbcad6-af0b-4606-9df6-b349b6eb4013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:23.261226 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bbcad6-af0b-4606-9df6-b349b6eb4013 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:23.261503 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bbcad6-af0b-4606-9df6-b349b6eb4013 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:23.261774 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bbcad6-af0b-4606-9df6-b349b6eb4013 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:23.272606 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bbcad6-af0b-4606-9df6-b349b6eb4013 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:04:23.278041 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-59bbcad6-af0b-4606-9df6-b349b6eb4013 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 12:04:23.278041 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:04:23.278041 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:04:23.278041 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:04:23.278041 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:04:23.278041 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:04:23.278041 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:04:23.278041 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:04:23.278041 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:04:23.278041 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:04:23.278041 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:04:23.278041 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:04:23.278041 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:04:23.278041 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:04:23.278041 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:04:23.278041 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:04:23.283308 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:04:23.283308 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:04:23.283308 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:04:23.283308 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:04:23.283308 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:04:23.283308 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:04:23.283308 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:04:23.283308 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:04:23.283308 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:04:23.283308 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:04:23.283308 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:04:23.283308 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:04:23.283308 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:04:23.283308 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:04:23.283308 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:04:23.283308 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:04:23.284347 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:04:23.284347 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:04:23.284347 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:04:23.284347 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:04:23.284347 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:04:23.284347 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:04:23.284347 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:04:23.284347 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:04:23.284347 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:04:23.284347 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:04:23.284347 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 12:04:23.284347 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:04:23.284347 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 600/1208] 10.222.0.61 () {56 vars in 1102 bytes} [Fri Aug 26 12:04:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:04:23.285268 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d081fb-389b-44d5-8d9f-4d4bb8eb9cd0 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:23.334636 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:23.399430 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d081fb-389b-44d5-8d9f-4d4bb8eb9cd0 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:23.434913 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bfd9a4-c319-440a-b001-00727233db9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:23.435568 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bfd9a4-c319-440a-b001-00727233db9d None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:23.435848 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bfd9a4-c319-440a-b001-00727233db9d None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:23.436127 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bfd9a4-c319-440a-b001-00727233db9d None admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:23.436795 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bfd9a4-c319-440a-b001-00727233db9d None admin] RBAC: Adding query filter params (name=service) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:04:23.437097 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bfd9a4-c319-440a-b001-00727233db9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:23.438470 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bfd9a4-c319-440a-b001-00727233db9d None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:23.447300 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 609/1209] 10.222.0.61 () {56 vars in 1103 bytes} [Fri Aug 26 12:04:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:04:23.453825 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bbcad6-af0b-4606-9df6-b349b6eb4013 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:23.518475 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:23.582825 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bbcad6-af0b-4606-9df6-b349b6eb4013 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:23.623777 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79203ff9-da90-4db1-82e3-d18c2a69acc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:23.624406 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79203ff9-da90-4db1-82e3-d18c2a69acc9 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:23.624687 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79203ff9-da90-4db1-82e3-d18c2a69acc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:23.624959 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79203ff9-da90-4db1-82e3-d18c2a69acc9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:23.637289 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79203ff9-da90-4db1-82e3-d18c2a69acc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:23.638749 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79203ff9-da90-4db1-82e3-d18c2a69acc9 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:23.654583 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-79203ff9-da90-4db1-82e3-d18c2a69acc9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:04:23.654583 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:04:23.654583 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:04:23.654583 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:04:23.654583 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:04:23.654583 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:04:23.654583 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:04:23.654583 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:04:23.654583 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:04:23.654583 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:04:23.654583 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:04:23.654583 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:04:23.654583 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:04:23.654583 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:04:23.654583 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:04:23.654583 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:04:23.660829 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:04:23.660829 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:04:23.660829 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:04:23.660829 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:04:23.660829 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:04:23.660829 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:04:23.660829 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:04:23.660829 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:04:23.660829 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:04:23.660829 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:04:23.660829 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:04:23.660829 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:04:23.660829 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:04:23.660829 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:04:23.660829 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:04:23.660829 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:04:23.660829 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:04:23.661748 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:04:23.661748 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:04:23.661748 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:04:23.661748 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 601/1210] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:04:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 12:04:23.661996 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bfd9a4-c319-440a-b001-00727233db9d None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:23.782673 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:23.785944 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bfd9a4-c319-440a-b001-00727233db9d None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:23.824239 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929d9ff0-7dc4-408a-b73c-d0f51bfb0cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:23.824861 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d9ff0-7dc4-408a-b73c-d0f51bfb0cb2 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:23.825121 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d9ff0-7dc4-408a-b73c-d0f51bfb0cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:23.825431 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d9ff0-7dc4-408a-b73c-d0f51bfb0cb2 None admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:23.826104 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929d9ff0-7dc4-408a-b73c-d0f51bfb0cb2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:04:23.826403 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929d9ff0-7dc4-408a-b73c-d0f51bfb0cb2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:23.827907 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929d9ff0-7dc4-408a-b73c-d0f51bfb0cb2 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:23.843316 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 610/1211] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:04:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:04:23.849028 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79203ff9-da90-4db1-82e3-d18c2a69acc9 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:23.969713 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79203ff9-da90-4db1-82e3-d18c2a69acc9 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:23.972148 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:24.012983 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5bbab6-7669-402b-a145-c418752a7d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:24.013647 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5bbab6-7669-402b-a145-c418752a7d32 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:24.013931 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5bbab6-7669-402b-a145-c418752a7d32 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:24.014208 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5bbab6-7669-402b-a145-c418752a7d32 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:24.023545 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5bbab6-7669-402b-a145-c418752a7d32 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:24.025168 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5bbab6-7669-402b-a145-c418752a7d32 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:24.037588 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-8d5bbab6-7669-402b-a145-c418752a7d32 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 26 12:04:24.037588 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:04:24.037588 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:04:24.037588 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:04:24.037588 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:04:24.037588 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:04:24.037588 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:04:24.037588 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:04:24.037588 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:04:24.037588 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:04:24.037588 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:04:24.037588 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:04:24.037588 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 12:04:24.037588 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 12:04:24.037588 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 12:04:24.037588 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 12:04:24.038738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:04:24.038738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:04:24.038738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 12:04:24.038738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:04:24.038738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 12:04:24.038738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:04:24.038738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:04:24.038738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:04:24.038738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:04:24.038738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:04:24.038738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:04:24.038738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:04:24.038738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:04:24.038738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:04:24.038738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:04:24.038738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:04:24.038738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:04:24.038738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:04:24.039781 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:04:24.039781 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:04:24.039781 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 12:04:24.039781 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 12:04:24.039781 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 12:04:24.039781 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 12:04:24.039781 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 12:04:24.039781 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 12:04:24.039781 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 26 12:04:24.039781 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:04:24.039781 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 602/1212] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:04:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 12:04:24.048094 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929d9ff0-7dc4-408a-b73c-d0f51bfb0cb2 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:24.168873 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929d9ff0-7dc4-408a-b73c-d0f51bfb0cb2 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:24.210934 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d1b136-b544-481b-80cd-57ae64ef55a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:24.211618 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d1b136-b544-481b-80cd-57ae64ef55a4 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:24.211911 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d1b136-b544-481b-80cd-57ae64ef55a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:24.212281 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d1b136-b544-481b-80cd-57ae64ef55a4 None admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:24.213001 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d1b136-b544-481b-80cd-57ae64ef55a4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:04:24.213828 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d1b136-b544-481b-80cd-57ae64ef55a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:24.215784 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d1b136-b544-481b-80cd-57ae64ef55a4 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:24.238501 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 611/1213] 10.222.0.61 () {56 vars in 1143 bytes} [Fri Aug 26 12:04:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:04:24.244506 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5bbab6-7669-402b-a145-c418752a7d32 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:24.366984 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5bbab6-7669-402b-a145-c418752a7d32 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:24.413187 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9745e150-d887-4945-80ac-dcb0a976297a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:24.413960 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9745e150-d887-4945-80ac-dcb0a976297a None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:24.414352 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9745e150-d887-4945-80ac-dcb0a976297a None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:24.414673 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9745e150-d887-4945-80ac-dcb0a976297a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:24.451542 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:24.505160 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9745e150-d887-4945-80ac-dcb0a976297a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:24.506686 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9745e150-d887-4945-80ac-dcb0a976297a None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:24.635076 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 603/1214] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:04:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 12:04:24.641873 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d1b136-b544-481b-80cd-57ae64ef55a4 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:24.664032 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:24.772585 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d1b136-b544-481b-80cd-57ae64ef55a4 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:24.821187 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0487df2-beb0-427e-a53f-2b5c1702ac0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:24.822756 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0487df2-beb0-427e-a53f-2b5c1702ac0f None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:24.823299 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0487df2-beb0-427e-a53f-2b5c1702ac0f None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:24.823790 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0487df2-beb0-427e-a53f-2b5c1702ac0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:24.835249 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0487df2-beb0-427e-a53f-2b5c1702ac0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:24.837202 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0487df2-beb0-427e-a53f-2b5c1702ac0f None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:24.854575 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-c0487df2-beb0-427e-a53f-2b5c1702ac0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:04:24.854575 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:04:24.854575 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:04:24.854575 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:04:24.854575 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:04:24.854575 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:04:24.854575 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:04:24.854575 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:04:24.854575 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:04:24.854575 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:04:24.854575 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:04:24.854575 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:04:24.854575 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:04:24.854575 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:04:24.854575 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:04:24.854575 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:04:24.859311 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:04:24.859311 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:04:24.859311 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:04:24.859311 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:04:24.859311 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:04:24.859311 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:04:24.859311 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:04:24.859311 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:04:24.859311 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:04:24.859311 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:04:24.859311 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:04:24.859311 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:04:24.859311 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:04:24.859311 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:04:24.859311 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:04:24.859311 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:04:24.859311 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:04:24.860495 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:04:24.860495 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:04:24.860495 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:04:24.860495 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 612/1215] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:04:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:04:24.861704 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9745e150-d887-4945-80ac-dcb0a976297a None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:24.933470 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:24.974572 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9745e150-d887-4945-80ac-dcb0a976297a None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:25.016107 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18887918-c7d7-4691-a904-bbfc0759b4cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:25.016787 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18887918-c7d7-4691-a904-bbfc0759b4cd None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:25.017074 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18887918-c7d7-4691-a904-bbfc0759b4cd None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:25.017350 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18887918-c7d7-4691-a904-bbfc0759b4cd None admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:25.017992 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18887918-c7d7-4691-a904-bbfc0759b4cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:04:25.018295 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18887918-c7d7-4691-a904-bbfc0759b4cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:25.019777 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18887918-c7d7-4691-a904-bbfc0759b4cd None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:25.038142 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 604/1216] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:04:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:04:25.039814 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0487df2-beb0-427e-a53f-2b5c1702ac0f None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:25.117731 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:25.154749 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0487df2-beb0-427e-a53f-2b5c1702ac0f None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:25.196671 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ff45e6-33eb-4d2b-ac18-5add3df0165f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:25.197338 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff45e6-33eb-4d2b-ac18-5add3df0165f None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:25.198191 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff45e6-33eb-4d2b-ac18-5add3df0165f None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:25.198465 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff45e6-33eb-4d2b-ac18-5add3df0165f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:25.205860 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ff45e6-33eb-4d2b-ac18-5add3df0165f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:25.207435 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ff45e6-33eb-4d2b-ac18-5add3df0165f None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:25.220184 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-d4ff45e6-33eb-4d2b-ac18-5add3df0165f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:04:25.220184 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:04:25.220184 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:04:25.220184 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:04:25.220184 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:04:25.220184 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:04:25.220184 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:04:25.220184 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:04:25.220184 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:04:25.220184 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:04:25.220184 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:04:25.220184 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:04:25.220184 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 12:04:25.220184 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 12:04:25.220184 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 12:04:25.221447 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 12:04:25.221447 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:04:25.221447 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:04:25.221447 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:04:25.221447 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:04:25.221447 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:04:25.221447 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:04:25.221447 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:04:25.221447 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:04:25.221447 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:04:25.221447 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:04:25.221447 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:04:25.221447 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:04:25.221447 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:04:25.221447 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:04:25.221447 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:04:25.221447 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:04:25.222583 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 12:04:25.222583 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 12:04:25.222583 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 12:04:25.222583 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 12:04:25.222583 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 12:04:25.222583 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 12:04:25.222583 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:04:25.222583 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:04:25.222583 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 613/1217] 10.222.0.61 () {56 vars in 1111 bytes} [Fri Aug 26 12:04:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 26 12:04:25.228562 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18887918-c7d7-4691-a904-bbfc0759b4cd None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:25.347491 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18887918-c7d7-4691-a904-bbfc0759b4cd None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:25.370436 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:25.383035 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee9f4b9-3bf9-42b9-b5ba-8a1394a132cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:25.383975 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee9f4b9-3bf9-42b9-b5ba-8a1394a132cb None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:25.384315 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee9f4b9-3bf9-42b9-b5ba-8a1394a132cb None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:25.384694 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee9f4b9-3bf9-42b9-b5ba-8a1394a132cb None admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:25.385980 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee9f4b9-3bf9-42b9-b5ba-8a1394a132cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:04:25.386346 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee9f4b9-3bf9-42b9-b5ba-8a1394a132cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:25.387851 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee9f4b9-3bf9-42b9-b5ba-8a1394a132cb None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:25.402564 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 605/1218] 10.222.0.61 () {56 vars in 1148 bytes} [Fri Aug 26 12:04:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:04:25.409451 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ff45e6-33eb-4d2b-ac18-5add3df0165f None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:25.521931 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ff45e6-33eb-4d2b-ac18-5add3df0165f None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:25.551339 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:25.557374 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fbd77a-ffb0-4e7f-abc7-664a85fb96a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:25.558235 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fbd77a-ffb0-4e7f-abc7-664a85fb96a2 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:25.558533 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fbd77a-ffb0-4e7f-abc7-664a85fb96a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:25.558970 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fbd77a-ffb0-4e7f-abc7-664a85fb96a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:25.559893 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fbd77a-ffb0-4e7f-abc7-664a85fb96a2 None admin] RBAC: Adding query filter params (role.id=519347d2bd2c4dc1aadb0b9f8c3c8e3d, scope.project.id=ecd68423a83a427681d44dbf266fce6e, user.id=7346e4a29dfb4a0a9768597704be286c) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:04:25.560280 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fbd77a-ffb0-4e7f-abc7-664a85fb96a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:25.561695 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fbd77a-ffb0-4e7f-abc7-664a85fb96a2 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:25.567415 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 614/1219] 10.222.0.61 () {56 vars in 1374 bytes} [Fri Aug 26 12:04:25 2022] GET /identity/v3/role_assignments?user.id=7346e4a29dfb4a0a9768597704be286c&scope.project.id=ecd68423a83a427681d44dbf266fce6e&role.id=519347d2bd2c4dc1aadb0b9f8c3c8e3d => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:04:27.666442 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82240a7f-2119-4fbb-a543-e32154d76011 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:27.667022 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82240a7f-2119-4fbb-a543-e32154d76011 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:27.667022 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82240a7f-2119-4fbb-a543-e32154d76011 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:27.668506 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 606/1220] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:04:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:04:27.673932 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de055543-5d62-4e02-a105-66a08779852f None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:27.674345 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de055543-5d62-4e02-a105-66a08779852f None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:27.674611 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de055543-5d62-4e02-a105-66a08779852f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:27.694801 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-de055543-5d62-4e02-a105-66a08779852f None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:04:27.696172 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-de055543-5d62-4e02-a105-66a08779852f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:04:27.776872 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 615/1221] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:04:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 12:04:28.047370 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82240a7f-2119-4fbb-a543-e32154d76011 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:28.162638 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82240a7f-2119-4fbb-a543-e32154d76011 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:28.203104 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c3ba76-3dfe-43c9-b0a4-15efd4264435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:28.203753 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c3ba76-3dfe-43c9-b0a4-15efd4264435 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:28.204033 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c3ba76-3dfe-43c9-b0a4-15efd4264435 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:28.204303 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c3ba76-3dfe-43c9-b0a4-15efd4264435 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:28.215627 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c3ba76-3dfe-43c9-b0a4-15efd4264435 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:04:28.220482 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-c2c3ba76-3dfe-43c9-b0a4-15efd4264435 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 12:04:28.220482 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:04:28.220482 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:04:28.220482 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:04:28.220482 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:04:28.220482 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:04:28.220482 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:04:28.220482 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:04:28.220482 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:04:28.220482 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:04:28.220482 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:04:28.220482 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:04:28.220482 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:04:28.220482 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:04:28.220482 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:04:28.220482 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:04:28.221431 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:04:28.221431 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:04:28.221431 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:04:28.221431 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:04:28.221431 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:04:28.221431 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:04:28.221431 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:04:28.221431 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:04:28.221431 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:04:28.221431 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:04:28.221431 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:04:28.221431 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:04:28.221431 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:04:28.221431 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:04:28.221431 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:04:28.221431 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:04:28.222349 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:04:28.222349 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:04:28.222349 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:04:28.222349 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:04:28.222349 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:04:28.222349 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:04:28.222349 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:04:28.222349 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:04:28.222349 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:04:28.222349 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:04:28.222349 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 12:04:28.222349 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:04:28.222349 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 607/1222] 10.222.0.61 () {56 vars in 1102 bytes} [Fri Aug 26 12:04:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:04:28.227953 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de055543-5d62-4e02-a105-66a08779852f None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:28.332291 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de055543-5d62-4e02-a105-66a08779852f None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:28.363536 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:28.376965 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd83044c-a78e-4ac3-99b7-4041778aa15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:28.377593 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd83044c-a78e-4ac3-99b7-4041778aa15e None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:28.377859 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd83044c-a78e-4ac3-99b7-4041778aa15e None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:28.378125 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd83044c-a78e-4ac3-99b7-4041778aa15e None admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:28.378802 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd83044c-a78e-4ac3-99b7-4041778aa15e None admin] RBAC: Adding query filter params (name=service) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:04:28.379089 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd83044c-a78e-4ac3-99b7-4041778aa15e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:28.380482 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd83044c-a78e-4ac3-99b7-4041778aa15e None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:28.388748 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 616/1223] 10.222.0.61 () {56 vars in 1103 bytes} [Fri Aug 26 12:04:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:04:28.394418 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c3ba76-3dfe-43c9-b0a4-15efd4264435 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:28.440328 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:28.503523 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c3ba76-3dfe-43c9-b0a4-15efd4264435 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:28.540007 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5e8f07-2455-4b87-8c11-f1b1957d4611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:28.540626 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e8f07-2455-4b87-8c11-f1b1957d4611 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:28.540904 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e8f07-2455-4b87-8c11-f1b1957d4611 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:28.541748 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e8f07-2455-4b87-8c11-f1b1957d4611 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:28.552595 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5e8f07-2455-4b87-8c11-f1b1957d4611 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:28.554239 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5e8f07-2455-4b87-8c11-f1b1957d4611 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:28.569787 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-af5e8f07-2455-4b87-8c11-f1b1957d4611 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:04:28.569787 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:04:28.569787 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:04:28.569787 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:04:28.569787 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:04:28.569787 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:04:28.569787 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:04:28.569787 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:04:28.569787 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:04:28.569787 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:04:28.569787 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:04:28.569787 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:04:28.569787 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:04:28.569787 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:04:28.569787 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:04:28.569787 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:04:28.570834 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:04:28.570834 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:04:28.570834 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:04:28.570834 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:04:28.570834 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:04:28.570834 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:04:28.570834 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:04:28.570834 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:04:28.570834 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:04:28.570834 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:04:28.570834 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:04:28.570834 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:04:28.570834 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:04:28.570834 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:04:28.570834 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:04:28.570834 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:04:28.570834 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:04:28.571957 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:04:28.571957 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:04:28.571957 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:04:28.571957 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 608/1224] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:04:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:04:28.576712 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd83044c-a78e-4ac3-99b7-4041778aa15e None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:28.689970 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd83044c-a78e-4ac3-99b7-4041778aa15e None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:28.726036 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa786b80-353f-47d9-8aed-b6b9fbc86af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:28.726671 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa786b80-353f-47d9-8aed-b6b9fbc86af0 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:28.727071 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa786b80-353f-47d9-8aed-b6b9fbc86af0 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:28.727376 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa786b80-353f-47d9-8aed-b6b9fbc86af0 None admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:28.728011 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa786b80-353f-47d9-8aed-b6b9fbc86af0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:04:28.728374 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa786b80-353f-47d9-8aed-b6b9fbc86af0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:28.730795 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa786b80-353f-47d9-8aed-b6b9fbc86af0 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:28.748133 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 617/1225] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:04:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:04:28.754679 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5e8f07-2455-4b87-8c11-f1b1957d4611 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:28.853364 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:28.861216 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5e8f07-2455-4b87-8c11-f1b1957d4611 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:28.898615 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96d02e3-423e-45ba-834f-21f21f943686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:28.899410 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d02e3-423e-45ba-834f-21f21f943686 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:28.899764 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d02e3-423e-45ba-834f-21f21f943686 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:28.900109 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d02e3-423e-45ba-834f-21f21f943686 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:28.908862 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96d02e3-423e-45ba-834f-21f21f943686 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:28.910459 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96d02e3-423e-45ba-834f-21f21f943686 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:28.922526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-d96d02e3-423e-45ba-834f-21f21f943686 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 26 12:04:28.922526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:04:28.922526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:04:28.922526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:04:28.922526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:04:28.922526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:04:28.922526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:04:28.922526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:04:28.922526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:04:28.922526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:04:28.922526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:04:28.922526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:04:28.922526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 12:04:28.922526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 12:04:28.922526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 12:04:28.922526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 12:04:28.923849 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:04:28.923849 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:04:28.923849 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 12:04:28.923849 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:04:28.923849 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 12:04:28.923849 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:04:28.923849 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:04:28.923849 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:04:28.923849 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:04:28.923849 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:04:28.923849 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:04:28.923849 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:04:28.923849 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:04:28.923849 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:04:28.923849 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:04:28.923849 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:04:28.923849 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:04:28.923849 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:04:28.924825 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:04:28.924825 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:04:28.924825 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 12:04:28.924825 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 12:04:28.924825 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 12:04:28.924825 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 12:04:28.924825 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 12:04:28.924825 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 12:04:28.924825 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 26 12:04:28.924825 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:04:28.924825 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 609/1226] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:04:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:04:28.929787 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa786b80-353f-47d9-8aed-b6b9fbc86af0 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:28.978496 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:29.039643 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa786b80-353f-47d9-8aed-b6b9fbc86af0 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:29.079684 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b0d03-4c78-43bb-88e5-a19539fdc08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:29.080505 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b0d03-4c78-43bb-88e5-a19539fdc08f None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:29.080839 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b0d03-4c78-43bb-88e5-a19539fdc08f None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:29.081148 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b0d03-4c78-43bb-88e5-a19539fdc08f None admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:29.082200 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9b0d03-4c78-43bb-88e5-a19539fdc08f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:04:29.082546 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9b0d03-4c78-43bb-88e5-a19539fdc08f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:29.085643 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9b0d03-4c78-43bb-88e5-a19539fdc08f None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:29.111777 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 618/1227] 10.222.0.61 () {56 vars in 1143 bytes} [Fri Aug 26 12:04:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:04:29.117484 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96d02e3-423e-45ba-834f-21f21f943686 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:29.246726 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96d02e3-423e-45ba-834f-21f21f943686 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:29.284942 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f8aaf6-7822-493a-8d45-eb447e3d0836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:29.285749 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f8aaf6-7822-493a-8d45-eb447e3d0836 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:29.286036 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f8aaf6-7822-493a-8d45-eb447e3d0836 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:29.286333 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f8aaf6-7822-493a-8d45-eb447e3d0836 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:29.299332 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f8aaf6-7822-493a-8d45-eb447e3d0836 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:29.300782 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f8aaf6-7822-493a-8d45-eb447e3d0836 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:29.317243 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-41f8aaf6-7822-493a-8d45-eb447e3d0836 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:04:29.317243 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:04:29.317243 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:04:29.317243 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:04:29.317243 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:04:29.317243 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:04:29.317243 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:04:29.317243 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:04:29.317243 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:04:29.317243 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:04:29.317243 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:04:29.317243 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:04:29.317243 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:04:29.317243 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:04:29.317243 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:04:29.317243 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:04:29.318579 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:04:29.318579 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:04:29.318579 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:04:29.318579 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:04:29.318579 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:04:29.318579 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:04:29.318579 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:04:29.318579 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:04:29.318579 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:04:29.318579 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:04:29.318579 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:04:29.318579 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:04:29.318579 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:04:29.318579 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:04:29.318579 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:04:29.318579 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:04:29.318579 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:04:29.319797 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:04:29.319797 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:04:29.319797 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:04:29.319797 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 610/1228] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:04:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:04:29.325157 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b0d03-4c78-43bb-88e5-a19539fdc08f None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:29.443263 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b0d03-4c78-43bb-88e5-a19539fdc08f None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:29.446071 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:29.487885 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932d66ef-38c9-497a-bc06-1bf0c6f99025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:29.488596 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d66ef-38c9-497a-bc06-1bf0c6f99025 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:29.488910 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d66ef-38c9-497a-bc06-1bf0c6f99025 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:29.489286 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d66ef-38c9-497a-bc06-1bf0c6f99025 None admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:29.490054 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932d66ef-38c9-497a-bc06-1bf0c6f99025 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:04:29.490552 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932d66ef-38c9-497a-bc06-1bf0c6f99025 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:29.492068 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932d66ef-38c9-497a-bc06-1bf0c6f99025 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:29.505457 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 619/1229] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:04:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:04:29.512158 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f8aaf6-7822-493a-8d45-eb447e3d0836 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:29.521449 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:29.629298 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f8aaf6-7822-493a-8d45-eb447e3d0836 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:29.671573 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf363495-e777-4b74-881e-b3bc8bba5962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:29.672395 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf363495-e777-4b74-881e-b3bc8bba5962 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:29.672625 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf363495-e777-4b74-881e-b3bc8bba5962 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:29.672926 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf363495-e777-4b74-881e-b3bc8bba5962 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:29.680934 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf363495-e777-4b74-881e-b3bc8bba5962 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:29.683634 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf363495-e777-4b74-881e-b3bc8bba5962 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:29.696615 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-bf363495-e777-4b74-881e-b3bc8bba5962 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:04:29.696615 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:04:29.696615 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:04:29.696615 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:04:29.696615 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:04:29.696615 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:04:29.696615 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:04:29.696615 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:04:29.696615 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:04:29.696615 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:04:29.696615 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:04:29.696615 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:04:29.696615 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 12:04:29.696615 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 12:04:29.696615 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 12:04:29.698300 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 12:04:29.698300 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:04:29.698300 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:04:29.698300 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:04:29.698300 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:04:29.698300 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:04:29.698300 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:04:29.698300 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:04:29.698300 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:04:29.698300 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:04:29.698300 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:04:29.698300 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:04:29.698300 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:04:29.698300 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:04:29.698300 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:04:29.698300 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:04:29.698300 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:04:29.699413 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 12:04:29.699413 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 12:04:29.699413 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 12:04:29.699413 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 12:04:29.699413 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 12:04:29.699413 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 12:04:29.699413 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:04:29.699413 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:04:29.699413 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 611/1230] 10.222.0.61 () {56 vars in 1111 bytes} [Fri Aug 26 12:04:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:04:29.705866 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932d66ef-38c9-497a-bc06-1bf0c6f99025 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:29.830358 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932d66ef-38c9-497a-bc06-1bf0c6f99025 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:29.867427 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8d1531-df10-4f69-bf2a-6554eaf096c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:29.868063 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8d1531-df10-4f69-bf2a-6554eaf096c8 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:29.868362 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8d1531-df10-4f69-bf2a-6554eaf096c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:29.868761 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8d1531-df10-4f69-bf2a-6554eaf096c8 None admin] PATH_INFO: `/v3/projects` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:29.870119 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8d1531-df10-4f69-bf2a-6554eaf096c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:04:29.870499 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8d1531-df10-4f69-bf2a-6554eaf096c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:29.872289 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8d1531-df10-4f69-bf2a-6554eaf096c8 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:29.886901 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 620/1231] 10.222.0.61 () {56 vars in 1148 bytes} [Fri Aug 26 12:04:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:04:29.894384 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf363495-e777-4b74-881e-b3bc8bba5962 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:29.966439 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:30.005494 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf363495-e777-4b74-881e-b3bc8bba5962 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:30.044053 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69727527-e36b-4e79-9317-fd6d4be31248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:30.044803 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69727527-e36b-4e79-9317-fd6d4be31248 None admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:30.045084 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69727527-e36b-4e79-9317-fd6d4be31248 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:30.045358 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69727527-e36b-4e79-9317-fd6d4be31248 None admin] PATH_INFO: `/v3/projects/ecd68423a83a427681d44dbf266fce6e/users/7346e4a29dfb4a0a9768597704be286c/roles/519347d2bd2c4dc1aadb0b9f8c3c8e3d` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:30.060634 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69727527-e36b-4e79-9317-fd6d4be31248 None admin] RBAC: Authorizing `identity:create_grant(project_id=ecd68423a83a427681d44dbf266fce6e, user_id=7346e4a29dfb4a0a9768597704be286c, role_id=519347d2bd2c4dc1aadb0b9f8c3c8e3d)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:30.062465 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69727527-e36b-4e79-9317-fd6d4be31248 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:30.076124 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 612/1232] 10.222.0.61 () {58 vars in 1439 bytes} [Fri Aug 26 12:04:29 2022] PUT /identity/v3/projects/ecd68423a83a427681d44dbf266fce6e/users/7346e4a29dfb4a0a9768597704be286c/roles/519347d2bd2c4dc1aadb0b9f8c3c8e3d => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:04:32.127164 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aadf098-8900-40dd-a2a2-aec239fa4167 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:32.127761 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aadf098-8900-40dd-a2a2-aec239fa4167 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:32.127761 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aadf098-8900-40dd-a2a2-aec239fa4167 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:32.129218 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 621/1233] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:04:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 12:04:32.134772 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba605f7-61b7-4aaa-95b3-327d32fda0df None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:32.135969 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba605f7-61b7-4aaa-95b3-327d32fda0df None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:32.136080 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba605f7-61b7-4aaa-95b3-327d32fda0df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:32.157158 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-2ba605f7-61b7-4aaa-95b3-327d32fda0df None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:04:32.158342 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-2ba605f7-61b7-4aaa-95b3-327d32fda0df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:04:32.244835 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 613/1234] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:04:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 12:04:32.535065 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aadf098-8900-40dd-a2a2-aec239fa4167 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:32.562558 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:32.648100 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aadf098-8900-40dd-a2a2-aec239fa4167 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:32.690206 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad8db98-d06c-4321-a6f7-a491d8fad741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:32.690837 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad8db98-d06c-4321-a6f7-a491d8fad741 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:32.691117 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad8db98-d06c-4321-a6f7-a491d8fad741 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:32.691415 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad8db98-d06c-4321-a6f7-a491d8fad741 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:32.702897 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad8db98-d06c-4321-a6f7-a491d8fad741 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:04:32.707569 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-7ad8db98-d06c-4321-a6f7-a491d8fad741 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 26 12:04:32.707569 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:04:32.707569 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:04:32.707569 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:04:32.707569 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:04:32.707569 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:04:32.707569 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:04:32.707569 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:04:32.707569 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:04:32.707569 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:04:32.707569 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:04:32.707569 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:04:32.707569 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:04:32.707569 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:04:32.707569 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:04:32.707569 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:04:32.708719 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:04:32.708719 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:04:32.708719 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:04:32.708719 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:04:32.708719 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:04:32.708719 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:04:32.708719 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:04:32.708719 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:04:32.708719 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:04:32.708719 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:04:32.708719 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:04:32.708719 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:04:32.708719 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:04:32.708719 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:04:32.708719 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:04:32.708719 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:04:32.712370 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:04:32.712370 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:04:32.712370 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:04:32.712370 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:04:32.712370 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:04:32.712370 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:04:32.712370 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:04:32.712370 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:04:32.712370 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:04:32.712370 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:04:32.712370 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 26 12:04:32.712370 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:04:32.712370 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 622/1235] 10.222.0.61 () {56 vars in 1102 bytes} [Fri Aug 26 12:04:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:04:32.714779 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba605f7-61b7-4aaa-95b3-327d32fda0df None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:32.774500 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:32.838580 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba605f7-61b7-4aaa-95b3-327d32fda0df None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:32.877143 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308da398-d5f2-4ae7-b83c-b8bcf182c24d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:32.877854 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308da398-d5f2-4ae7-b83c-b8bcf182c24d None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:32.878358 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308da398-d5f2-4ae7-b83c-b8bcf182c24d None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:32.878634 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308da398-d5f2-4ae7-b83c-b8bcf182c24d None admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:32.879339 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308da398-d5f2-4ae7-b83c-b8bcf182c24d None admin] RBAC: Adding query filter params (name=service) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:04:32.879641 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308da398-d5f2-4ae7-b83c-b8bcf182c24d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:32.881199 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308da398-d5f2-4ae7-b83c-b8bcf182c24d None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:32.891508 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 614/1236] 10.222.0.61 () {56 vars in 1103 bytes} [Fri Aug 26 12:04:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:04:32.897579 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad8db98-d06c-4321-a6f7-a491d8fad741 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:32.988647 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:33.006217 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad8db98-d06c-4321-a6f7-a491d8fad741 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:33.043359 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb366f76-a71e-4279-84c6-86ee43205317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:33.043987 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb366f76-a71e-4279-84c6-86ee43205317 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:33.044274 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb366f76-a71e-4279-84c6-86ee43205317 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:33.044543 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb366f76-a71e-4279-84c6-86ee43205317 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:33.055895 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb366f76-a71e-4279-84c6-86ee43205317 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:33.057386 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb366f76-a71e-4279-84c6-86ee43205317 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:33.071926 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-cb366f76-a71e-4279-84c6-86ee43205317 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:04:33.071926 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:04:33.071926 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:04:33.071926 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:04:33.071926 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:04:33.071926 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:04:33.071926 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:04:33.071926 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:04:33.071926 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:04:33.071926 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:04:33.071926 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:04:33.071926 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:04:33.071926 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:04:33.071926 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:04:33.071926 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:04:33.071926 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:04:33.073203 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:04:33.073203 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:04:33.073203 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:04:33.073203 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:04:33.073203 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:04:33.073203 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:04:33.073203 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:04:33.073203 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:04:33.073203 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:04:33.073203 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:04:33.073203 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:04:33.073203 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:04:33.073203 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:04:33.073203 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:04:33.073203 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:04:33.073203 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:04:33.073203 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:04:33.074301 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:04:33.074301 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:04:33.074301 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:04:33.074301 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 623/1237] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:04:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:04:33.080698 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308da398-d5f2-4ae7-b83c-b8bcf182c24d None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:33.202250 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308da398-d5f2-4ae7-b83c-b8bcf182c24d None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:33.211763 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:33.239974 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c755b683-f315-4c3d-9c5e-75d728b4baea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:33.240376 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c755b683-f315-4c3d-9c5e-75d728b4baea None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:33.240661 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c755b683-f315-4c3d-9c5e-75d728b4baea None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:33.240933 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c755b683-f315-4c3d-9c5e-75d728b4baea None admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:33.241576 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c755b683-f315-4c3d-9c5e-75d728b4baea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:04:33.241878 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c755b683-f315-4c3d-9c5e-75d728b4baea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:33.243263 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c755b683-f315-4c3d-9c5e-75d728b4baea None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:33.256105 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 615/1238] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:04:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:04:33.263440 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb366f76-a71e-4279-84c6-86ee43205317 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:33.388507 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb366f76-a71e-4279-84c6-86ee43205317 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:33.426555 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c74134-c842-423c-8928-d7152f9314b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:33.427403 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c74134-c842-423c-8928-d7152f9314b8 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:33.427679 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c74134-c842-423c-8928-d7152f9314b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:33.427953 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c74134-c842-423c-8928-d7152f9314b8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:33.431105 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:33.437393 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c74134-c842-423c-8928-d7152f9314b8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:33.438850 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c74134-c842-423c-8928-d7152f9314b8 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:33.451556 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-73c74134-c842-423c-8928-d7152f9314b8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 26 12:04:33.451556 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:04:33.451556 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:04:33.451556 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:04:33.451556 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:04:33.451556 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:04:33.451556 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:04:33.451556 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:04:33.451556 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:04:33.451556 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:04:33.451556 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:04:33.451556 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:04:33.451556 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 12:04:33.451556 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 12:04:33.451556 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 12:04:33.451556 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 12:04:33.453445 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:04:33.453445 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:04:33.453445 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 12:04:33.453445 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:04:33.453445 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 12:04:33.453445 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:04:33.453445 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:04:33.453445 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:04:33.453445 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:04:33.453445 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:04:33.453445 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:04:33.453445 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:04:33.453445 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:04:33.453445 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:04:33.453445 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:04:33.453445 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:04:33.453445 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:04:33.453445 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:04:33.455127 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:04:33.455127 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:04:33.455127 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 12:04:33.455127 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 12:04:33.455127 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 12:04:33.455127 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 12:04:33.455127 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 12:04:33.455127 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 12:04:33.455127 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 26 12:04:33.455127 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:04:33.457409 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 624/1239] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:04:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:04:33.458683 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c755b683-f315-4c3d-9c5e-75d728b4baea None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:33.561891 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c755b683-f315-4c3d-9c5e-75d728b4baea None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:33.605945 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2497c058-8815-433e-956a-93987d6e56e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:33.606557 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2497c058-8815-433e-956a-93987d6e56e7 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:33.606842 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2497c058-8815-433e-956a-93987d6e56e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:33.607114 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2497c058-8815-433e-956a-93987d6e56e7 None admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:33.607876 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2497c058-8815-433e-956a-93987d6e56e7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:04:33.608182 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2497c058-8815-433e-956a-93987d6e56e7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:33.609604 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2497c058-8815-433e-956a-93987d6e56e7 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:33.634318 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 616/1240] 10.222.0.61 () {56 vars in 1143 bytes} [Fri Aug 26 12:04:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:04:33.639864 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c74134-c842-423c-8928-d7152f9314b8 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:33.767915 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c74134-c842-423c-8928-d7152f9314b8 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:33.805012 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7a3969-1cea-4d63-ab8c-c3c224f49f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:33.805615 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a3969-1cea-4d63-ab8c-c3c224f49f8f None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:33.805895 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a3969-1cea-4d63-ab8c-c3c224f49f8f None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:33.806168 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a3969-1cea-4d63-ab8c-c3c224f49f8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:33.875162 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:33.882526 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7a3969-1cea-4d63-ab8c-c3c224f49f8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:33.884254 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7a3969-1cea-4d63-ab8c-c3c224f49f8f None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:34.001425 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 625/1241] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:04:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 12:04:34.007342 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2497c058-8815-433e-956a-93987d6e56e7 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:34.038441 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:34.121665 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2497c058-8815-433e-956a-93987d6e56e7 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:34.165912 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a767ce31-fc3f-4f92-8464-ad5b5851b72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:34.167097 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a767ce31-fc3f-4f92-8464-ad5b5851b72e None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:34.167441 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a767ce31-fc3f-4f92-8464-ad5b5851b72e None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:34.167785 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a767ce31-fc3f-4f92-8464-ad5b5851b72e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:34.181711 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a767ce31-fc3f-4f92-8464-ad5b5851b72e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:34.183829 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a767ce31-fc3f-4f92-8464-ad5b5851b72e None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:34.206123 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-a767ce31-fc3f-4f92-8464-ad5b5851b72e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:04:34.206123 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:04:34.206123 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:04:34.206123 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:04:34.206123 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:04:34.206123 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:04:34.206123 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:04:34.206123 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:04:34.206123 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:04:34.206123 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:04:34.206123 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:04:34.206123 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:04:34.206123 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:04:34.206123 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:04:34.206123 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:04:34.206123 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:04:34.216129 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:04:34.216129 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:04:34.216129 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:04:34.216129 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:04:34.216129 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:04:34.216129 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:04:34.216129 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:04:34.216129 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:04:34.216129 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:04:34.216129 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:04:34.216129 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:04:34.216129 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:04:34.216129 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:04:34.216129 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:04:34.216129 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:04:34.216129 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:04:34.216129 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:04:34.218118 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:04:34.218118 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:04:34.218118 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:04:34.218118 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 617/1242] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:04:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:04:34.218628 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7a3969-1cea-4d63-ab8c-c3c224f49f8f None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:34.330317 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:34.337357 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7a3969-1cea-4d63-ab8c-c3c224f49f8f None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:34.375272 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d214e087-a307-498e-8b41-7e7302ef9e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:34.375981 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d214e087-a307-498e-8b41-7e7302ef9e0a None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:34.376263 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d214e087-a307-498e-8b41-7e7302ef9e0a None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:34.376561 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d214e087-a307-498e-8b41-7e7302ef9e0a None admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:34.377382 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d214e087-a307-498e-8b41-7e7302ef9e0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:04:34.377744 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d214e087-a307-498e-8b41-7e7302ef9e0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:34.379214 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d214e087-a307-498e-8b41-7e7302ef9e0a None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:34.393551 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 626/1243] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:04:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:04:34.399364 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a767ce31-fc3f-4f92-8464-ad5b5851b72e None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:34.497893 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:34.521649 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a767ce31-fc3f-4f92-8464-ad5b5851b72e None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:34.559811 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf8d9a1-ceda-4094-b58a-d6795a6e1324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:34.560430 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf8d9a1-ceda-4094-b58a-d6795a6e1324 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:34.560705 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf8d9a1-ceda-4094-b58a-d6795a6e1324 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:34.560971 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf8d9a1-ceda-4094-b58a-d6795a6e1324 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:34.567649 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf8d9a1-ceda-4094-b58a-d6795a6e1324 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:34.569070 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf8d9a1-ceda-4094-b58a-d6795a6e1324 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:34.582523 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-fbf8d9a1-ceda-4094-b58a-d6795a6e1324 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:04:34.582523 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:04:34.582523 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:04:34.582523 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:04:34.582523 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:04:34.582523 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:04:34.582523 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:04:34.582523 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:04:34.582523 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:04:34.582523 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:04:34.582523 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:04:34.582523 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:04:34.582523 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 12:04:34.582523 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 12:04:34.582523 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 12:04:34.583769 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 12:04:34.583769 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:04:34.583769 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:04:34.583769 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:04:34.583769 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:04:34.583769 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:04:34.583769 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:04:34.583769 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:04:34.583769 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:04:34.583769 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:04:34.583769 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:04:34.583769 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:04:34.583769 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:04:34.583769 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:04:34.583769 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:04:34.583769 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:04:34.583769 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:04:34.584688 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 12:04:34.584688 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 12:04:34.584688 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 12:04:34.584688 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 12:04:34.584688 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 12:04:34.584688 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 12:04:34.584688 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:04:34.584688 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:04:34.584688 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 618/1244] 10.222.0.61 () {56 vars in 1111 bytes} [Fri Aug 26 12:04:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:04:34.591429 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d214e087-a307-498e-8b41-7e7302ef9e0a None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:34.720788 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d214e087-a307-498e-8b41-7e7302ef9e0a None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:34.758095 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a60e5a-0147-47c4-b24a-7c7d89eba797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:34.758985 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a60e5a-0147-47c4-b24a-7c7d89eba797 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:34.759296 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a60e5a-0147-47c4-b24a-7c7d89eba797 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:34.759578 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a60e5a-0147-47c4-b24a-7c7d89eba797 None admin] PATH_INFO: `/v3/projects` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:34.760981 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a60e5a-0147-47c4-b24a-7c7d89eba797 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:04:34.761301 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a60e5a-0147-47c4-b24a-7c7d89eba797 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:34.762887 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a60e5a-0147-47c4-b24a-7c7d89eba797 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:34.775427 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 627/1245] 10.222.0.61 () {56 vars in 1148 bytes} [Fri Aug 26 12:04:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:04:34.782542 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf8d9a1-ceda-4094-b58a-d6795a6e1324 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:34.895149 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf8d9a1-ceda-4094-b58a-d6795a6e1324 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:34.933031 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33442d61-4a7c-4305-aeb5-4a7dbf307afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:34.933722 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33442d61-4a7c-4305-aeb5-4a7dbf307afe None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:34.934028 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33442d61-4a7c-4305-aeb5-4a7dbf307afe None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:34.934302 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33442d61-4a7c-4305-aeb5-4a7dbf307afe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:34.935107 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33442d61-4a7c-4305-aeb5-4a7dbf307afe None admin] RBAC: Adding query filter params (role.id=519347d2bd2c4dc1aadb0b9f8c3c8e3d, scope.project.id=ecd68423a83a427681d44dbf266fce6e, user.id=7346e4a29dfb4a0a9768597704be286c) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:04:34.935447 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33442d61-4a7c-4305-aeb5-4a7dbf307afe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:34.936984 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33442d61-4a7c-4305-aeb5-4a7dbf307afe None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:34.943448 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 619/1246] 10.222.0.61 () {56 vars in 1374 bytes} [Fri Aug 26 12:04:34 2022] GET /identity/v3/role_assignments?user.id=7346e4a29dfb4a0a9768597704be286c&scope.project.id=ecd68423a83a427681d44dbf266fce6e&role.id=519347d2bd2c4dc1aadb0b9f8c3c8e3d => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:04:37.015677 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9525125d-88f3-4290-8594-d1e3e3c8e4d3 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:37.016193 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9525125d-88f3-4290-8594-d1e3e3c8e4d3 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:37.016193 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9525125d-88f3-4290-8594-d1e3e3c8e4d3 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:37.017664 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 628/1247] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:04:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:04:37.023194 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b72604-36d8-4c85-be7c-ff2338db2178 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:37.023701 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b72604-36d8-4c85-be7c-ff2338db2178 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:37.023790 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b72604-36d8-4c85-be7c-ff2338db2178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:37.031146 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:37.043060 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-99b72604-36d8-4c85-be7c-ff2338db2178 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:04:37.044267 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-99b72604-36d8-4c85-be7c-ff2338db2178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:04:37.118085 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 620/1248] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:04:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 12:04:37.394378 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9525125d-88f3-4290-8594-d1e3e3c8e4d3 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:37.414221 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:37.511443 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9525125d-88f3-4290-8594-d1e3e3c8e4d3 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:37.554686 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e99b9d-dfa6-4d10-b888-1855ecb290fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:37.555382 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e99b9d-dfa6-4d10-b888-1855ecb290fa None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:37.555684 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e99b9d-dfa6-4d10-b888-1855ecb290fa None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:37.555979 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e99b9d-dfa6-4d10-b888-1855ecb290fa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:37.568260 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e99b9d-dfa6-4d10-b888-1855ecb290fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:04:37.574844 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-55e99b9d-dfa6-4d10-b888-1855ecb290fa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 12:04:37.574844 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:04:37.574844 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:04:37.574844 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:04:37.574844 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:04:37.574844 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:04:37.574844 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:04:37.574844 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:04:37.574844 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:04:37.574844 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:04:37.574844 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:04:37.574844 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:04:37.574844 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:04:37.574844 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:04:37.574844 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:04:37.574844 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:04:37.576035 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:04:37.576035 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:04:37.576035 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:04:37.576035 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:04:37.576035 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:04:37.576035 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:04:37.576035 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:04:37.576035 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:04:37.576035 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:04:37.576035 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:04:37.576035 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:04:37.576035 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:04:37.576035 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:04:37.576035 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:04:37.576035 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:04:37.576035 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:04:37.576908 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:04:37.576908 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:04:37.576908 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:04:37.576908 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:04:37.576908 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:04:37.576908 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:04:37.576908 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:04:37.576908 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:04:37.576908 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:04:37.576908 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:04:37.576908 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 12:04:37.576908 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:04:37.576908 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 629/1249] 10.222.0.61 () {56 vars in 1096 bytes} [Fri Aug 26 12:04:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:04:37.582152 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b72604-36d8-4c85-be7c-ff2338db2178 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:37.692976 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b72604-36d8-4c85-be7c-ff2338db2178 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:37.728393 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:37.734460 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c20c05-9af3-4c3c-8745-52158398b421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:37.735081 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c20c05-9af3-4c3c-8745-52158398b421 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:37.735458 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c20c05-9af3-4c3c-8745-52158398b421 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:37.735734 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c20c05-9af3-4c3c-8745-52158398b421 None admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:37.736400 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c20c05-9af3-4c3c-8745-52158398b421 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:04:37.736842 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c20c05-9af3-4c3c-8745-52158398b421 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:37.738363 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c20c05-9af3-4c3c-8745-52158398b421 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:37.749686 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 621/1250] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:04:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:04:37.755910 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e99b9d-dfa6-4d10-b888-1855ecb290fa None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:37.845891 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:37.873303 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e99b9d-dfa6-4d10-b888-1855ecb290fa None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:37.911624 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e05fd4-3a08-444a-8090-734a465f6a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:37.912260 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e05fd4-3a08-444a-8090-734a465f6a09 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:37.912588 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e05fd4-3a08-444a-8090-734a465f6a09 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:37.912829 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e05fd4-3a08-444a-8090-734a465f6a09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:37.925745 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e05fd4-3a08-444a-8090-734a465f6a09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:37.927657 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e05fd4-3a08-444a-8090-734a465f6a09 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:37.949175 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-58e05fd4-3a08-444a-8090-734a465f6a09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:04:37.949175 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:04:37.949175 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:04:37.949175 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:04:37.949175 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:04:37.949175 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:04:37.949175 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:04:37.949175 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:04:37.949175 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:04:37.949175 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:04:37.949175 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:04:37.949175 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:04:37.949175 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:04:37.949175 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:04:37.949175 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:04:37.949175 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:04:37.950347 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:04:37.950347 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:04:37.950347 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:04:37.950347 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:04:37.950347 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:04:37.950347 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:04:37.950347 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:04:37.950347 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:04:37.950347 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:04:37.950347 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:04:37.950347 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:04:37.950347 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:04:37.950347 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:04:37.950347 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:04:37.950347 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:04:37.950347 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:04:37.950347 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:04:37.951454 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:04:37.951454 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:04:37.951454 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:04:37.951454 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 630/1251] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:04:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:04:37.956805 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c20c05-9af3-4c3c-8745-52158398b421 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:38.068855 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c20c05-9af3-4c3c-8745-52158398b421 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:38.104996 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539a6977-ad6e-46b2-8e1a-44b4e6907c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:38.105603 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a6977-ad6e-46b2-8e1a-44b4e6907c82 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:38.105883 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a6977-ad6e-46b2-8e1a-44b4e6907c82 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:38.106155 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a6977-ad6e-46b2-8e1a-44b4e6907c82 None admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:38.106801 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539a6977-ad6e-46b2-8e1a-44b4e6907c82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:04:38.107100 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539a6977-ad6e-46b2-8e1a-44b4e6907c82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:38.108573 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539a6977-ad6e-46b2-8e1a-44b4e6907c82 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:38.120980 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 622/1252] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:04:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:04:38.127271 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e05fd4-3a08-444a-8090-734a465f6a09 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:38.255713 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e05fd4-3a08-444a-8090-734a465f6a09 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:38.287858 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:38.298062 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07daec7-68c6-4336-a5f0-a0d792ead09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:38.298828 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07daec7-68c6-4336-a5f0-a0d792ead09a None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:38.299163 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07daec7-68c6-4336-a5f0-a0d792ead09a None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:38.299476 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07daec7-68c6-4336-a5f0-a0d792ead09a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:38.307742 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07daec7-68c6-4336-a5f0-a0d792ead09a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:38.309374 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07daec7-68c6-4336-a5f0-a0d792ead09a None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:38.322173 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-a07daec7-68c6-4336-a5f0-a0d792ead09a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 26 12:04:38.322173 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:04:38.322173 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:04:38.322173 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:04:38.322173 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:04:38.322173 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:04:38.322173 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:04:38.322173 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:04:38.322173 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:04:38.322173 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:04:38.322173 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:04:38.322173 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:04:38.322173 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 12:04:38.322173 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 12:04:38.322173 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 12:04:38.322173 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 12:04:38.323322 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:04:38.323322 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:04:38.323322 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 12:04:38.323322 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:04:38.323322 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 12:04:38.323322 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:04:38.323322 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:04:38.323322 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:04:38.323322 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:04:38.323322 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:04:38.323322 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:04:38.323322 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:04:38.323322 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:04:38.323322 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:04:38.323322 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:04:38.323322 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:04:38.323322 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:04:38.323322 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:04:38.324279 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:04:38.324279 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:04:38.324279 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 12:04:38.324279 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 12:04:38.324279 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 12:04:38.324279 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 12:04:38.324279 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 12:04:38.324279 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 12:04:38.324279 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 26 12:04:38.324279 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:04:38.324279 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 631/1253] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:04:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:04:38.330023 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539a6977-ad6e-46b2-8e1a-44b4e6907c82 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:38.395638 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:38.444233 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539a6977-ad6e-46b2-8e1a-44b4e6907c82 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:38.479563 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d39da0-9796-4dce-b929-7081eb6aba42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:38.480187 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d39da0-9796-4dce-b929-7081eb6aba42 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:38.480532 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d39da0-9796-4dce-b929-7081eb6aba42 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:38.480884 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d39da0-9796-4dce-b929-7081eb6aba42 None admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:38.481619 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d39da0-9796-4dce-b929-7081eb6aba42 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:04:38.481926 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d39da0-9796-4dce-b929-7081eb6aba42 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:38.483411 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d39da0-9796-4dce-b929-7081eb6aba42 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:38.502749 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 623/1254] 10.222.0.61 () {56 vars in 1143 bytes} [Fri Aug 26 12:04:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:04:38.508828 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07daec7-68c6-4336-a5f0-a0d792ead09a None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:38.624551 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07daec7-68c6-4336-a5f0-a0d792ead09a None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:38.664840 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038ac003-f3ef-41a8-ae5a-443cd2e31051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:38.665446 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ac003-f3ef-41a8-ae5a-443cd2e31051 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:38.665722 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ac003-f3ef-41a8-ae5a-443cd2e31051 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:38.665992 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ac003-f3ef-41a8-ae5a-443cd2e31051 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:38.731914 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:38.751853 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038ac003-f3ef-41a8-ae5a-443cd2e31051 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:38.753334 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038ac003-f3ef-41a8-ae5a-443cd2e31051 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:38.862229 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 632/1255] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:04:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 12:04:38.868121 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d39da0-9796-4dce-b929-7081eb6aba42 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:38.971083 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d39da0-9796-4dce-b929-7081eb6aba42 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:38.984396 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:39.006730 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bcedcb-7ca3-48d7-b3ae-e90e043a9c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:39.007378 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bcedcb-7ca3-48d7-b3ae-e90e043a9c1f None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:39.007657 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bcedcb-7ca3-48d7-b3ae-e90e043a9c1f None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:39.007927 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bcedcb-7ca3-48d7-b3ae-e90e043a9c1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:39.019388 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bcedcb-7ca3-48d7-b3ae-e90e043a9c1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:39.020816 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bcedcb-7ca3-48d7-b3ae-e90e043a9c1f None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:39.036126 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-11bcedcb-7ca3-48d7-b3ae-e90e043a9c1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:04:39.036126 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:04:39.036126 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:04:39.036126 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:04:39.036126 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:04:39.036126 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:04:39.036126 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:04:39.036126 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:04:39.036126 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:04:39.036126 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:04:39.036126 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:04:39.036126 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:04:39.036126 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:04:39.036126 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:04:39.036126 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:04:39.036126 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:04:39.037201 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:04:39.037201 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:04:39.037201 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:04:39.037201 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:04:39.037201 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:04:39.037201 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:04:39.037201 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:04:39.037201 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:04:39.037201 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:04:39.037201 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:04:39.037201 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:04:39.037201 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:04:39.037201 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:04:39.037201 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:04:39.037201 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:04:39.037201 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:04:39.037201 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:04:39.038122 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:04:39.038122 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:04:39.038122 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:04:39.038122 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 624/1256] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:04:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:04:39.043337 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038ac003-f3ef-41a8-ae5a-443cd2e31051 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:39.138979 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:39.154655 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038ac003-f3ef-41a8-ae5a-443cd2e31051 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:39.197896 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dc7b87-c766-43be-890f-8d714089aa5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:39.199878 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc7b87-c766-43be-890f-8d714089aa5f None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:39.200175 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc7b87-c766-43be-890f-8d714089aa5f None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:39.200460 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc7b87-c766-43be-890f-8d714089aa5f None admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:39.201129 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dc7b87-c766-43be-890f-8d714089aa5f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:04:39.201433 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dc7b87-c766-43be-890f-8d714089aa5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:39.202817 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dc7b87-c766-43be-890f-8d714089aa5f None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:39.217159 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 633/1257] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:04:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:04:39.223081 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bcedcb-7ca3-48d7-b3ae-e90e043a9c1f None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:39.335431 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bcedcb-7ca3-48d7-b3ae-e90e043a9c1f None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:39.372113 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a4f7b8-5079-4d86-8e39-ab62a6b426f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:39.372736 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a4f7b8-5079-4d86-8e39-ab62a6b426f1 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:39.373020 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a4f7b8-5079-4d86-8e39-ab62a6b426f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:39.373290 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a4f7b8-5079-4d86-8e39-ab62a6b426f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:39.380013 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a4f7b8-5079-4d86-8e39-ab62a6b426f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:39.381458 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a4f7b8-5079-4d86-8e39-ab62a6b426f1 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:39.393612 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-c1a4f7b8-5079-4d86-8e39-ab62a6b426f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:04:39.393612 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:04:39.393612 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:04:39.393612 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:04:39.393612 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:04:39.393612 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:04:39.393612 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:04:39.393612 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:04:39.393612 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:04:39.393612 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:04:39.393612 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:04:39.393612 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:04:39.393612 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 12:04:39.393612 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 12:04:39.393612 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 12:04:39.394744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 12:04:39.394744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:04:39.394744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:04:39.394744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:04:39.394744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:04:39.394744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:04:39.394744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:04:39.394744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:04:39.394744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:04:39.394744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:04:39.394744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:04:39.394744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:04:39.394744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:04:39.394744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:04:39.394744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:04:39.394744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:04:39.394744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:04:39.395846 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 12:04:39.395846 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 12:04:39.395846 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 12:04:39.395846 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 12:04:39.395846 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 12:04:39.395846 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 12:04:39.395846 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:04:39.395846 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:04:39.395846 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 625/1258] 10.222.0.61 () {56 vars in 1111 bytes} [Fri Aug 26 12:04:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:04:39.401792 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dc7b87-c766-43be-890f-8d714089aa5f None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:39.506334 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dc7b87-c766-43be-890f-8d714089aa5f None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:39.544973 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d541c6e-cf7d-402d-bb4a-d71c35941337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:39.545727 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d541c6e-cf7d-402d-bb4a-d71c35941337 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:39.546103 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d541c6e-cf7d-402d-bb4a-d71c35941337 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:39.546501 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d541c6e-cf7d-402d-bb4a-d71c35941337 None admin] PATH_INFO: `/v3/projects` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:39.547342 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d541c6e-cf7d-402d-bb4a-d71c35941337 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:04:39.547647 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d541c6e-cf7d-402d-bb4a-d71c35941337 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:39.549077 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d541c6e-cf7d-402d-bb4a-d71c35941337 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:39.562041 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 634/1259] 10.222.0.61 () {56 vars in 1148 bytes} [Fri Aug 26 12:04:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:04:39.569033 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a4f7b8-5079-4d86-8e39-ab62a6b426f1 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:39.596627 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:39.677105 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a4f7b8-5079-4d86-8e39-ab62a6b426f1 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:39.713988 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a162363f-29ec-4d9c-865f-386b6143f5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:39.714778 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a162363f-29ec-4d9c-865f-386b6143f5bb None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:39.715033 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a162363f-29ec-4d9c-865f-386b6143f5bb None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:39.715332 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a162363f-29ec-4d9c-865f-386b6143f5bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:39.716365 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a162363f-29ec-4d9c-865f-386b6143f5bb None admin] RBAC: Adding query filter params (role.id=70ec1ba484714c8ca3adb8fe5845107c, scope.project.id=ecd68423a83a427681d44dbf266fce6e, user.id=7346e4a29dfb4a0a9768597704be286c) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:04:39.716672 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a162363f-29ec-4d9c-865f-386b6143f5bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:39.718177 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a162363f-29ec-4d9c-865f-386b6143f5bb None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:39.723855 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 626/1260] 10.222.0.61 () {56 vars in 1374 bytes} [Fri Aug 26 12:04:39 2022] GET /identity/v3/role_assignments?user.id=7346e4a29dfb4a0a9768597704be286c&scope.project.id=ecd68423a83a427681d44dbf266fce6e&role.id=70ec1ba484714c8ca3adb8fe5845107c => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:04:41.674712 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7117cb-050b-4d98-9da3-909cbd3f5927 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:41.675264 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7117cb-050b-4d98-9da3-909cbd3f5927 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:41.675264 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7117cb-050b-4d98-9da3-909cbd3f5927 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:41.676619 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 635/1261] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:04:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:04:41.681587 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c335c45-e447-4666-b5ea-9fab70c302a3 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:41.681986 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c335c45-e447-4666-b5ea-9fab70c302a3 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:41.682139 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c335c45-e447-4666-b5ea-9fab70c302a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:41.700749 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-1c335c45-e447-4666-b5ea-9fab70c302a3 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:04:41.701909 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-1c335c45-e447-4666-b5ea-9fab70c302a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:04:41.767620 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 627/1262] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:04:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 12:04:42.034042 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7117cb-050b-4d98-9da3-909cbd3f5927 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:42.044817 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:42.132725 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7117cb-050b-4d98-9da3-909cbd3f5927 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:42.171539 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c140b-a056-45fa-9e33-0c42dcd61c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:42.172336 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c140b-a056-45fa-9e33-0c42dcd61c41 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:42.172723 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c140b-a056-45fa-9e33-0c42dcd61c41 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:42.173102 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c140b-a056-45fa-9e33-0c42dcd61c41 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:42.186435 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2c140b-a056-45fa-9e33-0c42dcd61c41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:04:42.191250 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-5e2c140b-a056-45fa-9e33-0c42dcd61c41 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 12:04:42.191250 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:04:42.191250 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:04:42.191250 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:04:42.191250 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:04:42.191250 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:04:42.191250 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:04:42.191250 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:04:42.191250 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:04:42.191250 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:04:42.191250 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:04:42.191250 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:04:42.191250 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:04:42.191250 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:04:42.191250 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:04:42.191250 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:04:42.192627 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:04:42.192627 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:04:42.192627 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:04:42.192627 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:04:42.192627 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:04:42.192627 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:04:42.192627 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:04:42.192627 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:04:42.192627 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:04:42.192627 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:04:42.192627 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:04:42.192627 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:04:42.192627 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:04:42.192627 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:04:42.192627 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:04:42.192627 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:04:42.193660 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:04:42.193660 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:04:42.193660 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:04:42.193660 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:04:42.193660 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:04:42.193660 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:04:42.193660 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:04:42.193660 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:04:42.193660 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:04:42.193660 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:04:42.193660 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 12:04:42.193660 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:04:42.193660 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 636/1263] 10.222.0.61 () {56 vars in 1096 bytes} [Fri Aug 26 12:04:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:04:42.199816 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c335c45-e447-4666-b5ea-9fab70c302a3 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:42.220890 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:42.301162 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c335c45-e447-4666-b5ea-9fab70c302a3 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:42.334796 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07fb33f-027b-45e6-957c-7c7ba496b89c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:42.335407 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07fb33f-027b-45e6-957c-7c7ba496b89c None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:42.335680 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07fb33f-027b-45e6-957c-7c7ba496b89c None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:42.335944 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07fb33f-027b-45e6-957c-7c7ba496b89c None admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:42.336588 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07fb33f-027b-45e6-957c-7c7ba496b89c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:04:42.336883 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07fb33f-027b-45e6-957c-7c7ba496b89c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:42.338430 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07fb33f-027b-45e6-957c-7c7ba496b89c None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:42.346110 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 628/1264] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:04:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:04:42.352022 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c140b-a056-45fa-9e33-0c42dcd61c41 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:42.417609 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:42.449723 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c140b-a056-45fa-9e33-0c42dcd61c41 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:42.481560 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3facd61a-971a-4a4e-9cc3-95b4aedffb57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:42.482163 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3facd61a-971a-4a4e-9cc3-95b4aedffb57 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:42.482456 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3facd61a-971a-4a4e-9cc3-95b4aedffb57 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:42.482702 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3facd61a-971a-4a4e-9cc3-95b4aedffb57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:42.492338 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3facd61a-971a-4a4e-9cc3-95b4aedffb57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:42.493755 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3facd61a-971a-4a4e-9cc3-95b4aedffb57 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:42.508558 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-3facd61a-971a-4a4e-9cc3-95b4aedffb57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:04:42.508558 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:04:42.508558 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:04:42.508558 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:04:42.508558 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:04:42.508558 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:04:42.508558 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:04:42.508558 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:04:42.508558 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:04:42.508558 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:04:42.508558 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:04:42.508558 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:04:42.508558 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:04:42.508558 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:04:42.508558 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:04:42.508558 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:04:42.514970 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:04:42.514970 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:04:42.514970 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:04:42.514970 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:04:42.514970 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:04:42.514970 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:04:42.514970 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:04:42.514970 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:04:42.514970 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:04:42.514970 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:04:42.514970 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:04:42.514970 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:04:42.514970 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:04:42.514970 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:04:42.514970 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:04:42.514970 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:04:42.514970 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:04:42.516162 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:04:42.516162 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:04:42.516162 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:04:42.516162 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 637/1265] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:04:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:04:42.516444 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07fb33f-027b-45e6-957c-7c7ba496b89c None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:42.593448 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:42.612608 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07fb33f-027b-45e6-957c-7c7ba496b89c None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:42.644782 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c47d09a-3f2b-40ac-94bc-c05e546046c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:42.645366 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c47d09a-3f2b-40ac-94bc-c05e546046c2 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:42.645640 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c47d09a-3f2b-40ac-94bc-c05e546046c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:42.645906 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c47d09a-3f2b-40ac-94bc-c05e546046c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:42.646528 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c47d09a-3f2b-40ac-94bc-c05e546046c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:04:42.646822 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c47d09a-3f2b-40ac-94bc-c05e546046c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:42.648189 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c47d09a-3f2b-40ac-94bc-c05e546046c2 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:42.659448 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 629/1266] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:04:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:04:42.665153 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3facd61a-971a-4a4e-9cc3-95b4aedffb57 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:42.762617 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3facd61a-971a-4a4e-9cc3-95b4aedffb57 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:42.781807 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:42.796511 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7766be6-1967-47cf-bb83-22915324e4c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:42.797129 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7766be6-1967-47cf-bb83-22915324e4c3 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:42.797403 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7766be6-1967-47cf-bb83-22915324e4c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:42.797670 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7766be6-1967-47cf-bb83-22915324e4c3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:42.804891 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7766be6-1967-47cf-bb83-22915324e4c3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:42.806310 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7766be6-1967-47cf-bb83-22915324e4c3 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:42.817544 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-a7766be6-1967-47cf-bb83-22915324e4c3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 26 12:04:42.817544 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:04:42.817544 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:04:42.817544 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:04:42.817544 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:04:42.817544 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:04:42.817544 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:04:42.817544 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:04:42.817544 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:04:42.817544 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:04:42.817544 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:04:42.817544 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:04:42.817544 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 12:04:42.817544 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 12:04:42.817544 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 12:04:42.817544 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 12:04:42.818642 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:04:42.818642 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:04:42.818642 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 12:04:42.818642 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:04:42.818642 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 12:04:42.818642 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:04:42.818642 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:04:42.818642 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:04:42.818642 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:04:42.818642 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:04:42.818642 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:04:42.818642 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:04:42.818642 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:04:42.818642 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:04:42.818642 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:04:42.818642 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:04:42.818642 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:04:42.818642 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:04:42.820062 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:04:42.820062 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:04:42.820062 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 12:04:42.820062 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 12:04:42.820062 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 12:04:42.820062 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 12:04:42.820062 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 12:04:42.820062 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 12:04:42.820062 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 26 12:04:42.820062 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:04:42.820062 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 638/1267] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:04:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:04:42.824502 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c47d09a-3f2b-40ac-94bc-c05e546046c2 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:42.920320 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c47d09a-3f2b-40ac-94bc-c05e546046c2 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:42.952066 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c783d1-a835-4b97-9e35-428c9e71c078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:42.952649 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c783d1-a835-4b97-9e35-428c9e71c078 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:42.952923 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c783d1-a835-4b97-9e35-428c9e71c078 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:42.953186 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c783d1-a835-4b97-9e35-428c9e71c078 None admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:42.953895 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c783d1-a835-4b97-9e35-428c9e71c078 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:04:42.954193 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c783d1-a835-4b97-9e35-428c9e71c078 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:42.955623 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c783d1-a835-4b97-9e35-428c9e71c078 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:42.964459 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:42.972706 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 630/1268] 10.222.0.61 () {56 vars in 1143 bytes} [Fri Aug 26 12:04:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:04:42.978223 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7766be6-1967-47cf-bb83-22915324e4c3 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:43.075261 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7766be6-1967-47cf-bb83-22915324e4c3 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:43.108067 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6dabc4-efff-4961-ad25-acc45a534d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:43.108672 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6dabc4-efff-4961-ad25-acc45a534d18 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:43.108951 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6dabc4-efff-4961-ad25-acc45a534d18 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:43.109224 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6dabc4-efff-4961-ad25-acc45a534d18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:43.119527 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6dabc4-efff-4961-ad25-acc45a534d18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:43.121006 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6dabc4-efff-4961-ad25-acc45a534d18 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:43.134726 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-cc6dabc4-efff-4961-ad25-acc45a534d18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:04:43.134726 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:04:43.134726 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:04:43.134726 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:04:43.134726 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:04:43.134726 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:04:43.134726 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:04:43.134726 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:04:43.134726 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:04:43.134726 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:04:43.134726 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:04:43.134726 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:04:43.134726 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:04:43.134726 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:04:43.134726 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:04:43.134726 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:04:43.135975 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:04:43.135975 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:04:43.135975 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:04:43.135975 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:04:43.135975 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:04:43.135975 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:04:43.135975 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:04:43.135975 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:04:43.135975 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:04:43.135975 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:04:43.135975 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:04:43.135975 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:04:43.135975 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:04:43.135975 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:04:43.135975 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:04:43.135975 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:04:43.135975 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:04:43.136915 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:04:43.136915 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:04:43.136915 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:04:43.136915 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 639/1269] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:04:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:04:43.141659 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c783d1-a835-4b97-9e35-428c9e71c078 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:43.248947 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c783d1-a835-4b97-9e35-428c9e71c078 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:43.282327 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e78efea-7f7c-4b57-87e3-b846f75f5d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:43.282925 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e78efea-7f7c-4b57-87e3-b846f75f5d2c None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:43.283230 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e78efea-7f7c-4b57-87e3-b846f75f5d2c None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:43.283512 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e78efea-7f7c-4b57-87e3-b846f75f5d2c None admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:43.284141 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e78efea-7f7c-4b57-87e3-b846f75f5d2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:04:43.284475 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e78efea-7f7c-4b57-87e3-b846f75f5d2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:43.285829 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e78efea-7f7c-4b57-87e3-b846f75f5d2c None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:43.296564 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 631/1270] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:04:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:04:43.302232 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6dabc4-efff-4961-ad25-acc45a534d18 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:43.334449 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:43.402629 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6dabc4-efff-4961-ad25-acc45a534d18 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:43.435333 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021c391f-99ea-4994-8fe2-2f18fae73b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:43.435949 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c391f-99ea-4994-8fe2-2f18fae73b28 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:43.436227 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c391f-99ea-4994-8fe2-2f18fae73b28 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:43.436495 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c391f-99ea-4994-8fe2-2f18fae73b28 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:43.442907 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021c391f-99ea-4994-8fe2-2f18fae73b28 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:43.444364 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021c391f-99ea-4994-8fe2-2f18fae73b28 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:43.455905 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-021c391f-99ea-4994-8fe2-2f18fae73b28 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:04:43.455905 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:04:43.455905 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:04:43.455905 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:04:43.455905 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:04:43.455905 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:04:43.455905 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:04:43.455905 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:04:43.455905 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:04:43.455905 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:04:43.455905 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:04:43.455905 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:04:43.455905 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 12:04:43.455905 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 12:04:43.455905 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 12:04:43.456964 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 12:04:43.456964 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:04:43.456964 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:04:43.456964 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:04:43.456964 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:04:43.456964 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:04:43.456964 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:04:43.456964 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:04:43.456964 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:04:43.456964 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:04:43.456964 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:04:43.456964 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:04:43.456964 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:04:43.456964 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:04:43.456964 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:04:43.456964 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:04:43.456964 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:04:43.457917 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 12:04:43.457917 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 12:04:43.457917 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 12:04:43.457917 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 12:04:43.457917 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 12:04:43.457917 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 12:04:43.457917 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:04:43.457917 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:04:43.457917 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 640/1271] 10.222.0.61 () {56 vars in 1111 bytes} [Fri Aug 26 12:04:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:04:43.462799 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e78efea-7f7c-4b57-87e3-b846f75f5d2c None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:43.519810 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:43.560572 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e78efea-7f7c-4b57-87e3-b846f75f5d2c None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:43.593332 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76c4a09-c51c-416a-96f7-887a7eccd463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:43.593920 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76c4a09-c51c-416a-96f7-887a7eccd463 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:43.594199 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76c4a09-c51c-416a-96f7-887a7eccd463 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:43.594472 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76c4a09-c51c-416a-96f7-887a7eccd463 None admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:43.595149 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76c4a09-c51c-416a-96f7-887a7eccd463 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:04:43.595480 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76c4a09-c51c-416a-96f7-887a7eccd463 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:43.596897 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76c4a09-c51c-416a-96f7-887a7eccd463 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:43.608533 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 632/1272] 10.222.0.61 () {56 vars in 1148 bytes} [Fri Aug 26 12:04:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:04:43.614387 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021c391f-99ea-4994-8fe2-2f18fae73b28 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:43.706918 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:43.716322 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021c391f-99ea-4994-8fe2-2f18fae73b28 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:43.749120 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad73235-4ada-412e-875e-1f2ea2d9b40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:43.749770 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad73235-4ada-412e-875e-1f2ea2d9b40c None admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:43.750042 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad73235-4ada-412e-875e-1f2ea2d9b40c None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:43.750309 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad73235-4ada-412e-875e-1f2ea2d9b40c None admin] PATH_INFO: `/v3/projects/ecd68423a83a427681d44dbf266fce6e/users/7346e4a29dfb4a0a9768597704be286c/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:43.752173 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad73235-4ada-412e-875e-1f2ea2d9b40c None admin] RBAC: Authorizing `identity:create_grant(project_id=ecd68423a83a427681d44dbf266fce6e, user_id=7346e4a29dfb4a0a9768597704be286c, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:43.753583 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad73235-4ada-412e-875e-1f2ea2d9b40c None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:43.767299 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 641/1273] 10.222.0.61 () {58 vars in 1439 bytes} [Fri Aug 26 12:04:43 2022] PUT /identity/v3/projects/ecd68423a83a427681d44dbf266fce6e/users/7346e4a29dfb4a0a9768597704be286c/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:04:45.621299 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22689c9-2607-4e60-9e52-30bfa9fc7b14 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:45.621870 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22689c9-2607-4e60-9e52-30bfa9fc7b14 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:45.621870 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22689c9-2607-4e60-9e52-30bfa9fc7b14 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:45.623156 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 633/1274] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:04:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:04:45.628228 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2079ffd2-c65f-432c-8ac9-ece87dbd7087 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:45.628647 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2079ffd2-c65f-432c-8ac9-ece87dbd7087 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:45.628779 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2079ffd2-c65f-432c-8ac9-ece87dbd7087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:45.646634 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-2079ffd2-c65f-432c-8ac9-ece87dbd7087 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:04:45.647830 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-2079ffd2-c65f-432c-8ac9-ece87dbd7087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:04:45.713218 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 642/1275] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:04:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 26 12:04:45.978189 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22689c9-2607-4e60-9e52-30bfa9fc7b14 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:46.077019 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22689c9-2607-4e60-9e52-30bfa9fc7b14 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:46.093757 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:46.111043 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f949613-9212-4adc-b3a5-e7fa591034c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:46.111688 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f949613-9212-4adc-b3a5-e7fa591034c2 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:46.111967 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f949613-9212-4adc-b3a5-e7fa591034c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:46.112236 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f949613-9212-4adc-b3a5-e7fa591034c2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:46.121869 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f949613-9212-4adc-b3a5-e7fa591034c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 26 12:04:46.126460 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-7f949613-9212-4adc-b3a5-e7fa591034c2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 12:04:46.126460 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:04:46.126460 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:04:46.126460 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:04:46.126460 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:04:46.126460 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:04:46.126460 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:04:46.126460 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:04:46.126460 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:04:46.126460 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:04:46.126460 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:04:46.126460 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:04:46.126460 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 26 12:04:46.126460 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 26 12:04:46.126460 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 26 12:04:46.126460 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 26 12:04:46.127625 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 26 12:04:46.127625 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 26 12:04:46.127625 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 26 12:04:46.127625 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 26 12:04:46.127625 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:04:46.127625 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:04:46.127625 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:04:46.127625 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:04:46.127625 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:04:46.127625 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:04:46.127625 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:04:46.127625 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:04:46.127625 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:04:46.127625 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:04:46.127625 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:04:46.127625 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:04:46.128719 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:04:46.128719 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:04:46.128719 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:04:46.128719 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:04:46.128719 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 26 12:04:46.128719 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 26 12:04:46.128719 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 26 12:04:46.128719 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 26 12:04:46.128719 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 26 12:04:46.128719 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 26 12:04:46.128719 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 26 12:04:46.128719 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:04:46.128719 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 634/1276] 10.222.0.61 () {56 vars in 1096 bytes} [Fri Aug 26 12:04:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:04:46.133791 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2079ffd2-c65f-432c-8ac9-ece87dbd7087 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:46.218557 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:46.238487 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2079ffd2-c65f-432c-8ac9-ece87dbd7087 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:46.272511 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b050a5a6-25a7-40c7-98f7-6478f7dd9667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:46.273106 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b050a5a6-25a7-40c7-98f7-6478f7dd9667 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:46.273384 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b050a5a6-25a7-40c7-98f7-6478f7dd9667 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:46.273653 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b050a5a6-25a7-40c7-98f7-6478f7dd9667 None admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:46.274305 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b050a5a6-25a7-40c7-98f7-6478f7dd9667 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:04:46.274606 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b050a5a6-25a7-40c7-98f7-6478f7dd9667 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:46.276733 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b050a5a6-25a7-40c7-98f7-6478f7dd9667 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:46.284500 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 643/1277] 10.222.0.61 () {56 vars in 1099 bytes} [Fri Aug 26 12:04:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:04:46.290152 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f949613-9212-4adc-b3a5-e7fa591034c2 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:46.386923 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f949613-9212-4adc-b3a5-e7fa591034c2 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:46.420032 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f32c40b-c65a-4e3b-832f-100d99f8a40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:46.420635 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f32c40b-c65a-4e3b-832f-100d99f8a40b None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:46.420907 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f32c40b-c65a-4e3b-832f-100d99f8a40b None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:46.421171 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f32c40b-c65a-4e3b-832f-100d99f8a40b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:46.431021 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f32c40b-c65a-4e3b-832f-100d99f8a40b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:46.432604 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f32c40b-c65a-4e3b-832f-100d99f8a40b None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:46.445928 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-8f32c40b-c65a-4e3b-832f-100d99f8a40b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:04:46.445928 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:04:46.445928 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:04:46.445928 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:04:46.445928 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:04:46.445928 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:04:46.445928 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:04:46.445928 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:04:46.445928 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:04:46.445928 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:04:46.445928 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:04:46.445928 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:04:46.445928 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:04:46.445928 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:04:46.445928 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:04:46.445928 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:04:46.447052 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:04:46.447052 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:04:46.447052 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:04:46.447052 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:04:46.447052 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:04:46.447052 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:04:46.447052 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:04:46.447052 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:04:46.447052 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:04:46.447052 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:04:46.447052 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:04:46.447052 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:04:46.447052 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:04:46.447052 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:04:46.447052 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:04:46.447052 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:04:46.447052 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:04:46.449033 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:04:46.449033 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:04:46.449033 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:04:46.449033 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 635/1278] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:04:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:04:46.453281 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b050a5a6-25a7-40c7-98f7-6478f7dd9667 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:46.550488 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b050a5a6-25a7-40c7-98f7-6478f7dd9667 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:46.582314 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d672672e-6864-4dc7-a76c-c18789562b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:46.582896 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d672672e-6864-4dc7-a76c-c18789562b31 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:46.583170 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d672672e-6864-4dc7-a76c-c18789562b31 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:46.583469 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d672672e-6864-4dc7-a76c-c18789562b31 None admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:46.584090 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d672672e-6864-4dc7-a76c-c18789562b31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:04:46.584386 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d672672e-6864-4dc7-a76c-c18789562b31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:46.585725 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d672672e-6864-4dc7-a76c-c18789562b31 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:46.596950 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 644/1279] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:04:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:04:46.602465 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f32c40b-c65a-4e3b-832f-100d99f8a40b None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:46.632805 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:46.699089 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f32c40b-c65a-4e3b-832f-100d99f8a40b None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:46.731899 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cc3f07-bee7-4cfc-bf7b-c35966406949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:46.732502 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc3f07-bee7-4cfc-bf7b-c35966406949 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:46.732778 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc3f07-bee7-4cfc-bf7b-c35966406949 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:46.733047 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc3f07-bee7-4cfc-bf7b-c35966406949 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:46.740608 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cc3f07-bee7-4cfc-bf7b-c35966406949 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:46.742051 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cc3f07-bee7-4cfc-bf7b-c35966406949 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:46.761526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-53cc3f07-bee7-4cfc-bf7b-c35966406949 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 26 12:04:46.761526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:04:46.761526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:04:46.761526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:04:46.761526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:04:46.761526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:04:46.761526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:04:46.761526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:04:46.761526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:04:46.761526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:04:46.761526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:04:46.761526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:04:46.761526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 26 12:04:46.761526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 26 12:04:46.761526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 26 12:04:46.761526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 26 12:04:46.762663 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:04:46.762663 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:04:46.762663 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 26 12:04:46.762663 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:04:46.762663 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 26 12:04:46.762663 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 26 12:04:46.762663 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:04:46.762663 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:04:46.762663 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:04:46.762663 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:04:46.762663 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:04:46.762663 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:04:46.762663 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:04:46.762663 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:04:46.762663 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:04:46.762663 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:04:46.762663 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:04:46.762663 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:04:46.763873 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:04:46.763873 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:04:46.763873 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 26 12:04:46.763873 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 26 12:04:46.763873 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 26 12:04:46.763873 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 26 12:04:46.763873 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 26 12:04:46.763873 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 26 12:04:46.763873 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 26 12:04:46.763873 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:04:46.763873 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 636/1280] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:04:46 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:04:46.768376 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d672672e-6864-4dc7-a76c-c18789562b31 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:46.772148 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:46.866608 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d672672e-6864-4dc7-a76c-c18789562b31 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:46.898654 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8476e47d-8aec-420d-8d0b-4347695fa069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:46.899262 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8476e47d-8aec-420d-8d0b-4347695fa069 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:46.899543 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8476e47d-8aec-420d-8d0b-4347695fa069 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:46.899811 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8476e47d-8aec-420d-8d0b-4347695fa069 None admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:46.901056 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8476e47d-8aec-420d-8d0b-4347695fa069 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:04:46.901349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8476e47d-8aec-420d-8d0b-4347695fa069 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:46.902761 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8476e47d-8aec-420d-8d0b-4347695fa069 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:46.919665 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 645/1281] 10.222.0.61 () {56 vars in 1143 bytes} [Fri Aug 26 12:04:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:04:46.925369 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cc3f07-bee7-4cfc-bf7b-c35966406949 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:47.012617 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:47.022215 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cc3f07-bee7-4cfc-bf7b-c35966406949 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:47.054412 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39937dac-dad9-4aae-b3df-82eaee625806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:47.054993 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39937dac-dad9-4aae-b3df-82eaee625806 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:47.055298 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39937dac-dad9-4aae-b3df-82eaee625806 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:47.055571 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39937dac-dad9-4aae-b3df-82eaee625806 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:47.119778 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39937dac-dad9-4aae-b3df-82eaee625806 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:47.121135 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39937dac-dad9-4aae-b3df-82eaee625806 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:47.210952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:47.221748 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 637/1282] 10.222.0.61 () {58 vars in 1282 bytes} [Fri Aug 26 12:04:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 26 12:04:47.227555 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8476e47d-8aec-420d-8d0b-4347695fa069 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:47.287510 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:47.325553 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8476e47d-8aec-420d-8d0b-4347695fa069 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:47.357932 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cb4cb4-f49b-4281-bda7-ee1a05093304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:47.358539 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb4cb4-f49b-4281-bda7-ee1a05093304 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:47.358812 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb4cb4-f49b-4281-bda7-ee1a05093304 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:47.359079 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb4cb4-f49b-4281-bda7-ee1a05093304 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:47.368687 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cb4cb4-f49b-4281-bda7-ee1a05093304 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:47.370230 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cb4cb4-f49b-4281-bda7-ee1a05093304 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:47.384207 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-25cb4cb4-f49b-4281-bda7-ee1a05093304 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:04:47.384207 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:04:47.384207 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:04:47.384207 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:04:47.384207 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:04:47.384207 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:04:47.384207 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:04:47.384207 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:04:47.384207 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:04:47.384207 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:04:47.384207 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:04:47.384207 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:04:47.384207 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 26 12:04:47.384207 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 26 12:04:47.384207 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 26 12:04:47.384207 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 26 12:04:47.385919 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:04:47.385919 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:04:47.385919 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:04:47.385919 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:04:47.385919 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:04:47.385919 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:04:47.385919 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:04:47.385919 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:04:47.385919 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:04:47.385919 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:04:47.385919 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:04:47.385919 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:04:47.385919 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:04:47.385919 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:04:47.385919 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:04:47.385919 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:04:47.385919 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 26 12:04:47.386977 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 26 12:04:47.386977 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 26 12:04:47.386977 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:04:47.386977 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 646/1283] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:04:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:04:47.391510 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39937dac-dad9-4aae-b3df-82eaee625806 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:47.486266 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39937dac-dad9-4aae-b3df-82eaee625806 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:47.519338 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3416a130-6c6a-4b1f-9059-b8ecc99391ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:47.519931 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416a130-6c6a-4b1f-9059-b8ecc99391ac None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:47.520209 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416a130-6c6a-4b1f-9059-b8ecc99391ac None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:47.520478 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416a130-6c6a-4b1f-9059-b8ecc99391ac None admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:47.521110 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3416a130-6c6a-4b1f-9059-b8ecc99391ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:04:47.521409 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3416a130-6c6a-4b1f-9059-b8ecc99391ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:47.522759 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3416a130-6c6a-4b1f-9059-b8ecc99391ac None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:47.533604 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 638/1284] 10.222.0.61 () {56 vars in 1109 bytes} [Fri Aug 26 12:04:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:04:47.539112 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cb4cb4-f49b-4281-bda7-ee1a05093304 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:47.636266 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cb4cb4-f49b-4281-bda7-ee1a05093304 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:47.649032 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:47.668326 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0d0455-d5a5-4515-9b5e-e74d33330aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:47.668924 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0d0455-d5a5-4515-9b5e-e74d33330aa6 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:47.669197 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0d0455-d5a5-4515-9b5e-e74d33330aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:47.669461 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0d0455-d5a5-4515-9b5e-e74d33330aa6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:47.675830 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0d0455-d5a5-4515-9b5e-e74d33330aa6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:47.677276 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0d0455-d5a5-4515-9b5e-e74d33330aa6 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:47.688588 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application [None req-ed0d0455-d5a5-4515-9b5e-e74d33330aa6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:04:47.688588 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:04:47.688588 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:04:47.688588 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:04:47.688588 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:04:47.688588 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:04:47.688588 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:04:47.688588 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:04:47.688588 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:04:47.688588 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:04:47.688588 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:04:47.688588 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:04:47.688588 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 26 12:04:47.688588 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 26 12:04:47.688588 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 26 12:04:47.689658 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 26 12:04:47.689658 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:04:47.689658 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:04:47.689658 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:04:47.689658 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:04:47.689658 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:04:47.689658 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:04:47.689658 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:04:47.689658 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application with Lock( Aug 26 12:04:47.689658 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:04:47.689658 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:04:47.689658 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:04:47.689658 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:04:47.689658 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:04:47.689658 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:04:47.689658 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:04:47.689658 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:04:47.690623 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 26 12:04:47.690623 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 26 12:04:47.690623 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 26 12:04:47.690623 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 26 12:04:47.690623 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 26 12:04:47.690623 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 26 12:04:47.690623 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 26 12:04:47.690623 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: ERROR keystone.server.flask.application Aug 26 12:04:47.690623 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 647/1285] 10.222.0.61 () {56 vars in 1111 bytes} [Fri Aug 26 12:04:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:04:47.695326 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3416a130-6c6a-4b1f-9059-b8ecc99391ac None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:47.747311 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:47.796619 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3416a130-6c6a-4b1f-9059-b8ecc99391ac None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:47.828682 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36b0501-ecfc-47f3-8bc0-546756e63084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:47.829271 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b0501-ecfc-47f3-8bc0-546756e63084 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:47.829545 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b0501-ecfc-47f3-8bc0-546756e63084 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:47.829809 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b0501-ecfc-47f3-8bc0-546756e63084 None admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:47.830481 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36b0501-ecfc-47f3-8bc0-546756e63084 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:04:47.830780 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36b0501-ecfc-47f3-8bc0-546756e63084 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:47.832207 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36b0501-ecfc-47f3-8bc0-546756e63084 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:47.843708 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 639/1286] 10.222.0.61 () {56 vars in 1148 bytes} [Fri Aug 26 12:04:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:04:47.849743 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0d0455-d5a5-4515-9b5e-e74d33330aa6 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:47.947460 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0d0455-d5a5-4515-9b5e-e74d33330aa6 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:47.979522 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6abb885-747d-4376-9254-76d77721dc70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:47.980109 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6abb885-747d-4376-9254-76d77721dc70 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:47.980383 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6abb885-747d-4376-9254-76d77721dc70 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:47.980649 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6abb885-747d-4376-9254-76d77721dc70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:47.981356 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6abb885-747d-4376-9254-76d77721dc70 None admin] RBAC: Adding query filter params (role.id=70ec1ba484714c8ca3adb8fe5845107c, scope.project.id=ecd68423a83a427681d44dbf266fce6e, user.id=7346e4a29dfb4a0a9768597704be286c) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:04:47.981654 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6abb885-747d-4376-9254-76d77721dc70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:47.983051 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6abb885-747d-4376-9254-76d77721dc70 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:47.988585 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 648/1287] 10.222.0.61 () {56 vars in 1374 bytes} [Fri Aug 26 12:04:47 2022] GET /identity/v3/role_assignments?user.id=7346e4a29dfb4a0a9768597704be286c&scope.project.id=ecd68423a83a427681d44dbf266fce6e&role.id=70ec1ba484714c8ca3adb8fe5845107c => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:04:49.888709 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700fe0ad-f75f-4fcf-a4c1-92f88c573f94 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:49.889257 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700fe0ad-f75f-4fcf-a4c1-92f88c573f94 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:49.889257 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700fe0ad-f75f-4fcf-a4c1-92f88c573f94 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:49.890567 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 640/1288] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:04:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:04:49.895814 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301271f0-96b0-47fe-aada-639e5444910d None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:49.896233 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301271f0-96b0-47fe-aada-639e5444910d None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:49.896368 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301271f0-96b0-47fe-aada-639e5444910d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:49.914131 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-301271f0-96b0-47fe-aada-639e5444910d None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:04:49.915331 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-301271f0-96b0-47fe-aada-639e5444910d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:04:49.944289 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:49.981700 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 649/1289] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:04:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 12:04:50.261766 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700fe0ad-f75f-4fcf-a4c1-92f88c573f94 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:50.360339 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700fe0ad-f75f-4fcf-a4c1-92f88c573f94 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:50.371064 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:50.393909 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d116eab-cf79-482c-96e7-1c6f7f935824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:50.394517 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d116eab-cf79-482c-96e7-1c6f7f935824 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:50.394793 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d116eab-cf79-482c-96e7-1c6f7f935824 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:50.395064 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d116eab-cf79-482c-96e7-1c6f7f935824 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:50.395667 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d116eab-cf79-482c-96e7-1c6f7f935824 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:50.397056 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d116eab-cf79-482c-96e7-1c6f7f935824 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:50.405915 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-2d116eab-cf79-482c-96e7-1c6f7f935824 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 12:04:50.405915 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:04:50.405915 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 12:04:50.405915 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 12:04:50.405915 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 12:04:50.405915 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 12:04:50.405915 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 12:04:50.405915 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 12:04:50.405915 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 12:04:50.405915 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:04:50.405915 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 12:04:50.405915 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:04:50.405915 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:04:50.405915 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:04:50.405915 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:04:50.405915 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:04:50.405915 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:04:50.407225 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:04:50.407225 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:04:50.407225 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:04:50.407225 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:04:50.407225 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:04:50.407225 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:04:50.407225 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 12:04:50.407225 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 12:04:50.407225 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 12:04:50.407225 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 12:04:50.407225 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:04:50.407225 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:04:50.407225 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:04:50.407225 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:04:50.407225 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:04:50.407225 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:04:50.408399 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:04:50.408399 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:04:50.408399 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:04:50.408399 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:04:50.408399 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:04:50.408399 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:04:50.408399 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:04:50.408399 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:04:50.408399 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:04:50.408399 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:04:50.408399 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 12:04:50.408399 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 12:04:50.408399 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 12:04:50.408399 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:04:50.408399 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 641/1290] 10.222.0.61 () {56 vars in 1117 bytes} [Fri Aug 26 12:04:50 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:04:50.412902 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301271f0-96b0-47fe-aada-639e5444910d None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:50.508714 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301271f0-96b0-47fe-aada-639e5444910d None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:50.542783 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4a4286-76ff-4839-979a-b6109c30c062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:50.543343 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a4286-76ff-4839-979a-b6109c30c062 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:50.543624 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a4286-76ff-4839-979a-b6109c30c062 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:50.543896 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a4286-76ff-4839-979a-b6109c30c062 None admin] PATH_INFO: `/v3/services` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:50.544519 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4a4286-76ff-4839-979a-b6109c30c062 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:04:50.544821 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4a4286-76ff-4839-979a-b6109c30c062 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:50.546282 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4a4286-76ff-4839-979a-b6109c30c062 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:50.550961 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 650/1291] 10.222.0.61 () {56 vars in 1116 bytes} [Fri Aug 26 12:04:50 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:04:50.556649 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d116eab-cf79-482c-96e7-1c6f7f935824 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:50.657137 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d116eab-cf79-482c-96e7-1c6f7f935824 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:50.689371 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79081cc-f4b2-4eaa-b87d-468846677040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:50.689957 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79081cc-f4b2-4eaa-b87d-468846677040 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:50.690237 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79081cc-f4b2-4eaa-b87d-468846677040 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:50.690507 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79081cc-f4b2-4eaa-b87d-468846677040 None admin] PATH_INFO: `/v3/services` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:50.691121 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79081cc-f4b2-4eaa-b87d-468846677040 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:04:50.691448 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79081cc-f4b2-4eaa-b87d-468846677040 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:50.692795 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79081cc-f4b2-4eaa-b87d-468846677040 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:50.697029 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 642/1292] 10.222.0.61 () {56 vars in 1116 bytes} [Fri Aug 26 12:04:50 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:04:52.558552 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdec819-3a22-40e6-87cc-4532115f0408 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:52.559077 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdec819-3a22-40e6-87cc-4532115f0408 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:52.559077 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdec819-3a22-40e6-87cc-4532115f0408 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:52.560457 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 651/1293] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:04:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:04:52.565495 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b6810-4885-4815-ac20-6c0b0a3d0780 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:52.565874 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b6810-4885-4815-ac20-6c0b0a3d0780 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:52.566047 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b6810-4885-4815-ac20-6c0b0a3d0780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:52.583584 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-8c1b6810-4885-4815-ac20-6c0b0a3d0780 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:04:52.584736 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-8c1b6810-4885-4815-ac20-6c0b0a3d0780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:04:52.614873 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:52.651912 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 643/1294] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:04:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 12:04:52.919801 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdec819-3a22-40e6-87cc-4532115f0408 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:52.950111 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:53.019182 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdec819-3a22-40e6-87cc-4532115f0408 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:53.051706 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a9beea-3b3f-441f-bf7b-90b49b66b310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:53.052295 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9beea-3b3f-441f-bf7b-90b49b66b310 None admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:53.052571 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9beea-3b3f-441f-bf7b-90b49b66b310 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:53.052840 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9beea-3b3f-441f-bf7b-90b49b66b310 None admin] PATH_INFO: `/v3/services` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:53.053476 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a9beea-3b3f-441f-bf7b-90b49b66b310 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:53.054829 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a9beea-3b3f-441f-bf7b-90b49b66b310 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:53.068286 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 652/1295] 10.222.0.61 () {60 vars in 1141 bytes} [Fri Aug 26 12:04:52 2022] POST /identity/v3/services => generated 246 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:04:54.991627 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87716ed0-6a4c-4121-a317-bf5d2ae46e20 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:54.992147 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87716ed0-6a4c-4121-a317-bf5d2ae46e20 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:54.992239 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87716ed0-6a4c-4121-a317-bf5d2ae46e20 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:54.993564 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 644/1296] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:04:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:04:54.999244 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3126645d-aeb4-48ed-966e-5c27ef035e59 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:54.999661 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3126645d-aeb4-48ed-966e-5c27ef035e59 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:54.999793 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3126645d-aeb4-48ed-966e-5c27ef035e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:55.018175 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-3126645d-aeb4-48ed-966e-5c27ef035e59 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:04:55.019373 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-3126645d-aeb4-48ed-966e-5c27ef035e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:04:55.089844 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 653/1297] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:04:54 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 12:04:55.368096 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87716ed0-6a4c-4121-a317-bf5d2ae46e20 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:55.466997 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87716ed0-6a4c-4121-a317-bf5d2ae46e20 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:55.500300 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f40c19-e903-4102-9a51-ce79a6d084fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:55.500916 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f40c19-e903-4102-9a51-ce79a6d084fb None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:55.501194 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f40c19-e903-4102-9a51-ce79a6d084fb None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:55.501466 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f40c19-e903-4102-9a51-ce79a6d084fb None admin] PATH_INFO: `/v3/services/placement` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:55.502040 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f40c19-e903-4102-9a51-ce79a6d084fb None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:55.503427 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f40c19-e903-4102-9a51-ce79a6d084fb None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:55.512422 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-27f40c19-e903-4102-9a51-ce79a6d084fb None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 12:04:55.512422 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:04:55.512422 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 12:04:55.512422 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 12:04:55.512422 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 12:04:55.512422 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 12:04:55.512422 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 12:04:55.512422 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 12:04:55.512422 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 12:04:55.512422 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:04:55.512422 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 12:04:55.512422 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:04:55.512422 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:04:55.512422 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:04:55.512422 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:04:55.512422 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:04:55.512422 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:04:55.513571 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:04:55.513571 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:04:55.513571 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:04:55.513571 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:04:55.513571 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:04:55.513571 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:04:55.513571 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 12:04:55.513571 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 12:04:55.513571 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 12:04:55.513571 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 12:04:55.513571 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:04:55.513571 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:04:55.513571 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:04:55.513571 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:04:55.513571 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:04:55.513571 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:04:55.514563 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:04:55.514563 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:04:55.514563 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:04:55.514563 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:04:55.514563 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:04:55.514563 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:04:55.514563 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:04:55.514563 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:04:55.514563 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:04:55.514563 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:04:55.514563 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 12:04:55.514563 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 12:04:55.514563 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 12:04:55.514563 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:04:55.514563 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 645/1298] 10.222.0.61 () {56 vars in 1117 bytes} [Fri Aug 26 12:04:55 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:04:55.519454 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3126645d-aeb4-48ed-966e-5c27ef035e59 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:55.537026 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:55.616557 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3126645d-aeb4-48ed-966e-5c27ef035e59 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:55.649612 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767bd93c-7446-4ff0-b9ae-37116da116f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:55.650636 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767bd93c-7446-4ff0-b9ae-37116da116f2 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:55.650915 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767bd93c-7446-4ff0-b9ae-37116da116f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:55.651184 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767bd93c-7446-4ff0-b9ae-37116da116f2 None admin] PATH_INFO: `/v3/services` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:55.651833 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767bd93c-7446-4ff0-b9ae-37116da116f2 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:04:55.652133 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767bd93c-7446-4ff0-b9ae-37116da116f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:55.653490 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767bd93c-7446-4ff0-b9ae-37116da116f2 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:55.658453 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 654/1299] 10.222.0.61 () {56 vars in 1116 bytes} [Fri Aug 26 12:04:55 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:04:55.664407 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f40c19-e903-4102-9a51-ce79a6d084fb None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:55.695918 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:55.764504 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f40c19-e903-4102-9a51-ce79a6d084fb None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:55.797623 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdb2620-66d7-4f03-9f85-f617f2f2b2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:55.798212 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb2620-66d7-4f03-9f85-f617f2f2b2fa None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:55.798496 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb2620-66d7-4f03-9f85-f617f2f2b2fa None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:55.798768 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb2620-66d7-4f03-9f85-f617f2f2b2fa None admin] PATH_INFO: `/v3/endpoints` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:55.799488 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdb2620-66d7-4f03-9f85-f617f2f2b2fa None admin] RBAC: Adding query filter params (interface=public, service_id=07018086b9db43f1a7044a045efceaad, region_id=RegionOne) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:04:55.799795 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdb2620-66d7-4f03-9f85-f617f2f2b2fa None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:55.801150 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdb2620-66d7-4f03-9f85-f617f2f2b2fa None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:55.806056 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 646/1300] 10.222.0.61 () {56 vars in 1251 bytes} [Fri Aug 26 12:04:55 2022] GET /identity/v3/endpoints?service_id=07018086b9db43f1a7044a045efceaad&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:04:55.811834 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767bd93c-7446-4ff0-b9ae-37116da116f2 None admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:55.890070 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:55.908584 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767bd93c-7446-4ff0-b9ae-37116da116f2 None admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:55.941006 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a7994b-0ca1-4b8e-a44f-c33f9946c299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:55.941593 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a7994b-0ca1-4b8e-a44f-c33f9946c299 None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:55.941870 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a7994b-0ca1-4b8e-a44f-c33f9946c299 None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:55.942140 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a7994b-0ca1-4b8e-a44f-c33f9946c299 None admin] PATH_INFO: `/v3/services` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:55.942729 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a7994b-0ca1-4b8e-a44f-c33f9946c299 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:55.944101 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a7994b-0ca1-4b8e-a44f-c33f9946c299 None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:55.950713 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 655/1301] 10.222.0.61 () {56 vars in 1087 bytes} [Fri Aug 26 12:04:55 2022] GET /identity/v3/services => generated 2191 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 12:04:57.854107 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e308f-b9c5-4b20-80f4-f7dbbc3bca45 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:57.854626 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e308f-b9c5-4b20-80f4-f7dbbc3bca45 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:57.854717 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e308f-b9c5-4b20-80f4-f7dbbc3bca45 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:57.856047 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 647/1302] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:04:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:04:57.861427 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca882ceb-dc0e-4f7e-984d-099f9f5c42aa None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:57.861846 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca882ceb-dc0e-4f7e-984d-099f9f5c42aa None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:57.861984 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca882ceb-dc0e-4f7e-984d-099f9f5c42aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:57.879857 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-ca882ceb-dc0e-4f7e-984d-099f9f5c42aa None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:04:57.881069 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-ca882ceb-dc0e-4f7e-984d-099f9f5c42aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:04:57.973985 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 656/1303] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:04:57 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 26 12:04:58.251513 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73e308f-b9c5-4b20-80f4-f7dbbc3bca45 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:58.343841 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:58.350875 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73e308f-b9c5-4b20-80f4-f7dbbc3bca45 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:58.384830 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e7de66-e468-40b0-8a3a-8baf464932d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:58.385442 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7de66-e468-40b0-8a3a-8baf464932d7 None admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:58.385718 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7de66-e468-40b0-8a3a-8baf464932d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:58.385988 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7de66-e468-40b0-8a3a-8baf464932d7 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:58.386562 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e7de66-e468-40b0-8a3a-8baf464932d7 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:58.387952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e7de66-e468-40b0-8a3a-8baf464932d7 None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:58.396334 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application [None req-54e7de66-e468-40b0-8a3a-8baf464932d7 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 12:04:58.396334 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:04:58.396334 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 26 12:04:58.396334 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 26 12:04:58.396334 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 26 12:04:58.396334 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 26 12:04:58.396334 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 26 12:04:58.396334 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 12:04:58.396334 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 12:04:58.396334 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:04:58.396334 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 26 12:04:58.396334 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:04:58.396334 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 26 12:04:58.396334 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 26 12:04:58.396334 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 26 12:04:58.396334 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 26 12:04:58.396334 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 26 12:04:58.397460 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 26 12:04:58.397460 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 26 12:04:58.397460 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 26 12:04:58.397460 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 26 12:04:58.397460 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 26 12:04:58.397460 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 26 12:04:58.397460 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 26 12:04:58.397460 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 26 12:04:58.397460 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 26 12:04:58.397460 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 26 12:04:58.397460 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 26 12:04:58.397460 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 26 12:04:58.397460 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 26 12:04:58.397460 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 26 12:04:58.397460 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 26 12:04:58.397460 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.get_or_create( Aug 26 12:04:58.398317 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 26 12:04:58.398317 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application with Lock( Aug 26 12:04:58.398317 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 26 12:04:58.398317 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self._enter() Aug 26 12:04:58.398317 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 26 12:04:58.398317 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 26 12:04:58.398317 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 26 12:04:58.398317 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application return self.creator() Aug 26 12:04:58.398317 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 26 12:04:58.398317 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application created_value = creator( Aug 26 12:04:58.398317 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 26 12:04:58.398317 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 26 12:04:58.398317 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 26 12:04:58.398317 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: ERROR keystone.server.flask.application Aug 26 12:04:58.398317 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 648/1304] 10.222.0.61 () {56 vars in 1117 bytes} [Fri Aug 26 12:04:58 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 26 12:04:58.404223 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca882ceb-dc0e-4f7e-984d-099f9f5c42aa None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:58.477973 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:04:58.502267 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca882ceb-dc0e-4f7e-984d-099f9f5c42aa None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:58.534687 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b581c9e1-1d3e-456b-a2d2-9d56531848ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:58.535313 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b581c9e1-1d3e-456b-a2d2-9d56531848ca None admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:58.535598 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b581c9e1-1d3e-456b-a2d2-9d56531848ca None admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:58.535870 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b581c9e1-1d3e-456b-a2d2-9d56531848ca None admin] PATH_INFO: `/v3/services` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:58.536491 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b581c9e1-1d3e-456b-a2d2-9d56531848ca None admin] RBAC: Adding query filter params (name=placement) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:04:58.536792 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b581c9e1-1d3e-456b-a2d2-9d56531848ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:58.538243 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b581c9e1-1d3e-456b-a2d2-9d56531848ca None admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:58.543797 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 657/1305] 10.222.0.61 () {56 vars in 1116 bytes} [Fri Aug 26 12:04:58 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:04:58.549569 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e7de66-e468-40b0-8a3a-8baf464932d7 None admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:04:58.646698 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e7de66-e468-40b0-8a3a-8baf464932d7 None admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:04:58.679860 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926bb49b-e275-4b5f-aa70-3354532dc5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:04:58.680446 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926bb49b-e275-4b5f-aa70-3354532dc5fa None admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:04:58.680726 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926bb49b-e275-4b5f-aa70-3354532dc5fa None admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:04:58.681000 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926bb49b-e275-4b5f-aa70-3354532dc5fa None admin] PATH_INFO: `/v3/endpoints` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:04:58.681708 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926bb49b-e275-4b5f-aa70-3354532dc5fa None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:04:58.683143 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926bb49b-e275-4b5f-aa70-3354532dc5fa None admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:04:58.701520 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 649/1306] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:04:58 2022] POST /identity/v3/endpoints => generated 331 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:05:08.540313 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=30 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:05:08.697719 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:06:02.711352 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b86038-27da-4616-b5fc-99eb0bf1f62e None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:06:02.713279 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b86038-27da-4616-b5fc-99eb0bf1f62e None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:06:02.713279 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b86038-27da-4616-b5fc-99eb0bf1f62e None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:06:02.719375 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf2dd6-b834-4ce8-910f-dba176d3d623 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:06:02.720238 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf2dd6-b834-4ce8-910f-dba176d3d623 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:06:02.720238 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf2dd6-b834-4ce8-910f-dba176d3d623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:06:02.722540 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 658/1307] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:06:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:06:02.797616 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-4edf2dd6-b834-4ce8-910f-dba176d3d623 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:06:02.798817 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-4edf2dd6-b834-4ce8-910f-dba176d3d623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:06:03.020306 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 650/1308] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:06:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:06:03.345620 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24947da-2f0a-4c6a-a962-7f49f01d4f80 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:06:03.346272 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24947da-2f0a-4c6a-a962-7f49f01d4f80 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:06:03.346762 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24947da-2f0a-4c6a-a962-7f49f01d4f80 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:06:03.348628 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 659/1309] 10.222.0.61 () {54 vars in 965 bytes} [Fri Aug 26 12:06:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:06:03.355435 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3955f74-85de-40ea-802b-caa4b25b6e06 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:06:03.355726 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3955f74-85de-40ea-802b-caa4b25b6e06 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:06:03.356001 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3955f74-85de-40ea-802b-caa4b25b6e06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:06:03.436152 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-c3955f74-85de-40ea-802b-caa4b25b6e06 None None] MFA Rules not processed for user `b0972dbd5cff4610b239326425e6351c`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:06:03.437432 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-c3955f74-85de-40ea-802b-caa4b25b6e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:06:03.459589 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:06:03.519672 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 651/1310] 10.222.0.61 () {58 vars in 1062 bytes} [Fri Aug 26 12:06:03 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:06:03.529493 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24947da-2f0a-4c6a-a962-7f49f01d4f80 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:06:03.531622 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-b24947da-2f0a-4c6a-a962-7f49f01d4f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:06:03.667319 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24947da-2f0a-4c6a-a962-7f49f01d4f80 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:06:03.759533 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7a5975-5d73-4cb5-8535-380e2d2a6784 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:06:03.760192 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a5975-5d73-4cb5-8535-380e2d2a6784 service neutron] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:06:03.760515 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a5975-5d73-4cb5-8535-380e2d2a6784 service neutron] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:06:03.760853 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a5975-5d73-4cb5-8535-380e2d2a6784 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:06:03.762640 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-2d7a5975-5d73-4cb5-8535-380e2d2a6784 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:06:03.829553 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:06:03.904141 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7a5975-5d73-4cb5-8535-380e2d2a6784 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:06:03.905941 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7a5975-5d73-4cb5-8535-380e2d2a6784 service neutron] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:06:03.932992 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 660/1311] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:06:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:06:06.147643 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8346ab-3fef-421c-b6f9-a70c4237c4ad None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:06:06.148238 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8346ab-3fef-421c-b6f9-a70c4237c4ad None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:06:06.148238 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8346ab-3fef-421c-b6f9-a70c4237c4ad None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:06:06.150442 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 652/1312] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:06:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:06:06.155909 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47972f8-901c-4fe3-93d3-21917b8beb81 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:06:06.156320 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47972f8-901c-4fe3-93d3-21917b8beb81 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:06:06.156485 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47972f8-901c-4fe3-93d3-21917b8beb81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:06:06.238961 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-a47972f8-901c-4fe3-93d3-21917b8beb81 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:06:06.240179 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-a47972f8-901c-4fe3-93d3-21917b8beb81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:06:06.344185 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 661/1313] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:06:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:06:06.628114 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8346ab-3fef-421c-b6f9-a70c4237c4ad None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:06:06.629641 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-ee8346ab-3fef-421c-b6f9-a70c4237c4ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:06:06.737814 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8346ab-3fef-421c-b6f9-a70c4237c4ad None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:06:06.808703 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:06:06.811646 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74814a6d-c6f2-4b52-9bfa-8866b235341c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:06:06.812412 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74814a6d-c6f2-4b52-9bfa-8866b235341c service neutron] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:06:06.812750 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74814a6d-c6f2-4b52-9bfa-8866b235341c service neutron] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:06:06.813197 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74814a6d-c6f2-4b52-9bfa-8866b235341c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:06:06.814827 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-74814a6d-c6f2-4b52-9bfa-8866b235341c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:06:06.921334 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74814a6d-c6f2-4b52-9bfa-8866b235341c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:06:06.922768 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74814a6d-c6f2-4b52-9bfa-8866b235341c service neutron] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:06:06.942385 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 653/1314] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:06:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:06:09.149544 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef1016-5bf1-4ac0-bf4c-ff074401f495 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:06:09.150076 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef1016-5bf1-4ac0-bf4c-ff074401f495 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:06:09.150275 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef1016-5bf1-4ac0-bf4c-ff074401f495 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:06:09.151719 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 662/1315] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:06:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:06:09.156807 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568750a-5f84-4d8a-9c01-bc15a79f2dbc None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:06:09.157228 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568750a-5f84-4d8a-9c01-bc15a79f2dbc None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:06:09.157543 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568750a-5f84-4d8a-9c01-bc15a79f2dbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:06:09.224480 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-e568750a-5f84-4d8a-9c01-bc15a79f2dbc None None] MFA Rules not processed for user `06c7a037a0fa485aa111a5e7e792882f`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:06:09.225684 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-e568750a-5f84-4d8a-9c01-bc15a79f2dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:06:09.284307 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:06:09.328525 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 654/1316] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:06:09 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:06:09.615396 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecef1016-5bf1-4ac0-bf4c-ff074401f495 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:06:09.616932 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-ecef1016-5bf1-4ac0-bf4c-ff074401f495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:06:09.627189 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:06:09.720119 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecef1016-5bf1-4ac0-bf4c-ff074401f495 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:06:09.794024 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acf51b1-a521-4269-8f69-c8da867c4c20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:06:09.794632 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf51b1-a521-4269-8f69-c8da867c4c20 service neutron] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:06:09.794911 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf51b1-a521-4269-8f69-c8da867c4c20 service neutron] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:06:09.795186 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf51b1-a521-4269-8f69-c8da867c4c20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:06:09.796816 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-2acf51b1-a521-4269-8f69-c8da867c4c20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:06:09.858136 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:06:09.905893 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acf51b1-a521-4269-8f69-c8da867c4c20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:06:09.907354 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acf51b1-a521-4269-8f69-c8da867c4c20 service neutron] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:06:09.925801 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 663/1317] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:06:09 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:06:12.391008 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa571df-7261-4e8e-afc5-fb15e876244f None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:06:12.391571 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa571df-7261-4e8e-afc5-fb15e876244f None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:06:12.391571 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa571df-7261-4e8e-afc5-fb15e876244f None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:06:12.392922 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 655/1318] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:06:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:06:12.398088 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4a8152-1292-41a3-9fd7-62b9b6ee59fc None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:06:12.398511 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4a8152-1292-41a3-9fd7-62b9b6ee59fc None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:06:12.398633 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4a8152-1292-41a3-9fd7-62b9b6ee59fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:06:12.469362 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-1e4a8152-1292-41a3-9fd7-62b9b6ee59fc None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:06:12.470543 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-1e4a8152-1292-41a3-9fd7-62b9b6ee59fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:06:12.563553 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 664/1319] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:06:12 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:06:12.860090 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e173fc-5a3f-4076-8b8f-4208f7ba2878 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:06:12.860370 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e173fc-5a3f-4076-8b8f-4208f7ba2878 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:06:12.860646 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e173fc-5a3f-4076-8b8f-4208f7ba2878 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:06:12.861975 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 656/1320] 10.222.0.61 () {54 vars in 965 bytes} [Fri Aug 26 12:06:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:06:12.868894 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5fbfb5-fda8-4771-b979-a12119ea1678 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:06:12.869176 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5fbfb5-fda8-4771-b979-a12119ea1678 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:06:12.869476 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5fbfb5-fda8-4771-b979-a12119ea1678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:06:12.891216 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:06:12.944999 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-5d5fbfb5-fda8-4771-b979-a12119ea1678 None None] MFA Rules not processed for user `b0972dbd5cff4610b239326425e6351c`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:06:12.946326 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-5d5fbfb5-fda8-4771-b979-a12119ea1678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:06:13.021192 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 665/1321] 10.222.0.61 () {58 vars in 1062 bytes} [Fri Aug 26 12:06:12 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:06:13.030642 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e173fc-5a3f-4076-8b8f-4208f7ba2878 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:06:13.032199 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-43e173fc-5a3f-4076-8b8f-4208f7ba2878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:06:13.132302 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e173fc-5a3f-4076-8b8f-4208f7ba2878 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:06:13.203781 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b29c54d-f316-4245-a9dc-c0de05b5c00a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:06:13.204316 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29c54d-f316-4245-a9dc-c0de05b5c00a service neutron] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:06:13.204595 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29c54d-f316-4245-a9dc-c0de05b5c00a service neutron] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:06:13.204868 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29c54d-f316-4245-a9dc-c0de05b5c00a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:06:13.206459 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-9b29c54d-f316-4245-a9dc-c0de05b5c00a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:06:13.214465 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:06:13.330401 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b29c54d-f316-4245-a9dc-c0de05b5c00a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:06:13.331872 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b29c54d-f316-4245-a9dc-c0de05b5c00a service neutron] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:06:13.349835 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 657/1322] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:06:13 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:06:16.395353 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ffb827-f6e4-4097-be7f-62c0d68913af None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:06:16.396013 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ffb827-f6e4-4097-be7f-62c0d68913af None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:06:16.396013 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ffb827-f6e4-4097-be7f-62c0d68913af None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:06:16.397250 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 666/1323] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:06:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:06:16.403150 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d113700-74ef-49c0-ac05-4f74d29d68e7 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:06:16.403614 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d113700-74ef-49c0-ac05-4f74d29d68e7 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:06:16.403945 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d113700-74ef-49c0-ac05-4f74d29d68e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:06:16.471515 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-7d113700-74ef-49c0-ac05-4f74d29d68e7 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:06:16.472728 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-7d113700-74ef-49c0-ac05-4f74d29d68e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:06:16.531889 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:06:16.571838 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 658/1324] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:06:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:06:16.860272 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ffb827-f6e4-4097-be7f-62c0d68913af None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:06:16.861791 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-e4ffb827-f6e4-4097-be7f-62c0d68913af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:06:16.964513 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ffb827-f6e4-4097-be7f-62c0d68913af None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:06:16.976536 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:06:17.039277 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aef8bac-81d0-45ed-a3d1-b5ca5c2e9f49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:06:17.040311 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef8bac-81d0-45ed-a3d1-b5ca5c2e9f49 service neutron] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:06:17.040595 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef8bac-81d0-45ed-a3d1-b5ca5c2e9f49 service neutron] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:06:17.040870 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef8bac-81d0-45ed-a3d1-b5ca5c2e9f49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:06:17.042465 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-5aef8bac-81d0-45ed-a3d1-b5ca5c2e9f49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:06:17.151133 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aef8bac-81d0-45ed-a3d1-b5ca5c2e9f49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:06:17.152593 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aef8bac-81d0-45ed-a3d1-b5ca5c2e9f49 service neutron] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:06:17.171867 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 667/1325] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:06:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:06:19.151273 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eade4a9-bdf7-476a-9a00-b0a67e08b932 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:06:19.151840 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eade4a9-bdf7-476a-9a00-b0a67e08b932 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:06:19.151840 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eade4a9-bdf7-476a-9a00-b0a67e08b932 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:06:19.153163 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 659/1326] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:06:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:06:19.158567 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207439e1-2d39-46f5-9306-5a5485932f9b None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:06:19.159020 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207439e1-2d39-46f5-9306-5a5485932f9b None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:06:19.159132 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207439e1-2d39-46f5-9306-5a5485932f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:06:19.209216 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:06:19.228899 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-207439e1-2d39-46f5-9306-5a5485932f9b None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:06:19.230099 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-207439e1-2d39-46f5-9306-5a5485932f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:06:19.325708 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 668/1327] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:06:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:06:19.620594 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eade4a9-bdf7-476a-9a00-b0a67e08b932 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:06:19.622138 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-6eade4a9-bdf7-476a-9a00-b0a67e08b932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:06:19.735351 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eade4a9-bdf7-476a-9a00-b0a67e08b932 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:06:19.812808 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54998d6d-237c-4c92-bb49-dc7069252f81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:06:19.813413 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54998d6d-237c-4c92-bb49-dc7069252f81 service neutron] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:06:19.813688 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54998d6d-237c-4c92-bb49-dc7069252f81 service neutron] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:06:19.813956 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54998d6d-237c-4c92-bb49-dc7069252f81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:06:19.815574 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-54998d6d-237c-4c92-bb49-dc7069252f81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:06:19.832771 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:06:19.928570 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54998d6d-237c-4c92-bb49-dc7069252f81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:06:19.930036 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54998d6d-237c-4c92-bb49-dc7069252f81 service neutron] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:06:19.948364 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 660/1328] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:06:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:06:21.897397 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23866a-71ea-4aa3-aa43-83571abef181 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:06:21.897972 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23866a-71ea-4aa3-aa43-83571abef181 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:06:21.897972 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23866a-71ea-4aa3-aa43-83571abef181 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:06:21.900202 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 669/1329] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:06:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:06:21.906621 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770713b-b499-49a9-a796-74587bfd4f47 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:06:21.907946 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770713b-b499-49a9-a796-74587bfd4f47 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:06:21.908226 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770713b-b499-49a9-a796-74587bfd4f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:06:21.983324 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-f770713b-b499-49a9-a796-74587bfd4f47 None None] MFA Rules not processed for user `06c7a037a0fa485aa111a5e7e792882f`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:06:21.984581 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-f770713b-b499-49a9-a796-74587bfd4f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:06:22.053858 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:06:22.090108 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 661/1330] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:06:21 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:06:22.374090 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23866a-71ea-4aa3-aa43-83571abef181 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:06:22.375704 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-6d23866a-71ea-4aa3-aa43-83571abef181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:06:22.486787 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23866a-71ea-4aa3-aa43-83571abef181 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:06:22.506186 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:06:22.560944 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2519554-6dcc-4de4-9392-4e57a93f2aad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:06:22.561482 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2519554-6dcc-4de4-9392-4e57a93f2aad service neutron] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:06:22.561763 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2519554-6dcc-4de4-9392-4e57a93f2aad service neutron] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:06:22.562040 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2519554-6dcc-4de4-9392-4e57a93f2aad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:06:22.563681 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-c2519554-6dcc-4de4-9392-4e57a93f2aad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:06:22.673675 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2519554-6dcc-4de4-9392-4e57a93f2aad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:06:22.675139 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2519554-6dcc-4de4-9392-4e57a93f2aad service neutron] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:06:22.695386 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 670/1331] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:06:22 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:06:24.862755 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942fdc68-de03-4271-bc86-14b15796fd59 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:06:24.863273 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942fdc68-de03-4271-bc86-14b15796fd59 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:06:24.863568 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942fdc68-de03-4271-bc86-14b15796fd59 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:06:24.864654 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 662/1332] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:06:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:06:24.870440 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c5219-52aa-4b89-a034-43d1b249aa7c None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:06:24.870899 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c5219-52aa-4b89-a034-43d1b249aa7c None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:06:24.871006 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c5219-52aa-4b89-a034-43d1b249aa7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:06:24.935371 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:06:24.938464 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-248c5219-52aa-4b89-a034-43d1b249aa7c None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:06:24.940182 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-248c5219-52aa-4b89-a034-43d1b249aa7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:06:25.035155 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 671/1333] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:06:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:06:25.317972 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942fdc68-de03-4271-bc86-14b15796fd59 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:06:25.319596 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-942fdc68-de03-4271-bc86-14b15796fd59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:06:25.420744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942fdc68-de03-4271-bc86-14b15796fd59 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:06:25.490375 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac94a61-628f-48b7-87f5-b75680aea0b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:06:25.490966 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac94a61-628f-48b7-87f5-b75680aea0b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:06:25.491259 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac94a61-628f-48b7-87f5-b75680aea0b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:06:25.491529 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac94a61-628f-48b7-87f5-b75680aea0b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:06:25.493215 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-3ac94a61-628f-48b7-87f5-b75680aea0b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:06:25.519054 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:06:25.598690 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac94a61-628f-48b7-87f5-b75680aea0b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:06:25.600067 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac94a61-628f-48b7-87f5-b75680aea0b2 service neutron] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:06:25.617498 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 663/1334] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:06:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:06:27.553616 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bab924-4108-4013-9bbe-3f3931e6e2bc None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:06:27.554179 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bab924-4108-4013-9bbe-3f3931e6e2bc None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:06:27.554179 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bab924-4108-4013-9bbe-3f3931e6e2bc None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:06:27.556246 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 672/1335] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:06:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:06:27.561318 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8bb70a-d814-426c-a329-187dfe9492e7 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:06:27.561730 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8bb70a-d814-426c-a329-187dfe9492e7 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:06:27.561869 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8bb70a-d814-426c-a329-187dfe9492e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:06:27.625805 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-0d8bb70a-d814-426c-a329-187dfe9492e7 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:06:27.626978 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-0d8bb70a-d814-426c-a329-187dfe9492e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:06:27.699196 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:06:27.721163 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 664/1336] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:06:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:06:28.003452 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bab924-4108-4013-9bbe-3f3931e6e2bc None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:06:28.004972 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-43bab924-4108-4013-9bbe-3f3931e6e2bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:06:28.109429 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bab924-4108-4013-9bbe-3f3931e6e2bc None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:06:28.136909 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:06:28.182340 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e4885-e61f-4b16-9933-31029d717ae4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:06:28.182954 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e4885-e61f-4b16-9933-31029d717ae4 service neutron] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:06:28.183235 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e4885-e61f-4b16-9933-31029d717ae4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:06:28.183505 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e4885-e61f-4b16-9933-31029d717ae4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:06:28.185080 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-de0e4885-e61f-4b16-9933-31029d717ae4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:06:28.292071 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0e4885-e61f-4b16-9933-31029d717ae4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:06:28.293411 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0e4885-e61f-4b16-9933-31029d717ae4 service neutron] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:06:28.310629 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 673/1337] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:06:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:06:30.547571 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f35eb7c-2d10-4afa-93e1-0af9004ed369 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:06:30.548124 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f35eb7c-2d10-4afa-93e1-0af9004ed369 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:06:30.548124 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f35eb7c-2d10-4afa-93e1-0af9004ed369 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:06:30.549454 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 665/1338] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:06:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:06:30.555398 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41728e2e-6496-4b92-abed-2d657b2e2147 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:06:30.555825 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41728e2e-6496-4b92-abed-2d657b2e2147 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:06:30.555957 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41728e2e-6496-4b92-abed-2d657b2e2147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:06:30.620633 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-41728e2e-6496-4b92-abed-2d657b2e2147 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:06:30.621822 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-41728e2e-6496-4b92-abed-2d657b2e2147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:06:30.629868 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:06:30.717603 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 674/1339] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:06:30 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:06:31.002187 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f35eb7c-2d10-4afa-93e1-0af9004ed369 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:06:31.003833 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-9f35eb7c-2d10-4afa-93e1-0af9004ed369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:06:31.105428 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f35eb7c-2d10-4afa-93e1-0af9004ed369 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:06:31.176987 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18651ac-daae-457c-b623-caea31705db2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:06:31.177586 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18651ac-daae-457c-b623-caea31705db2 service neutron] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:06:31.177862 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18651ac-daae-457c-b623-caea31705db2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:06:31.178131 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18651ac-daae-457c-b623-caea31705db2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:06:31.180383 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-d18651ac-daae-457c-b623-caea31705db2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:06:31.214626 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:06:31.285367 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18651ac-daae-457c-b623-caea31705db2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:06:31.286703 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18651ac-daae-457c-b623-caea31705db2 service neutron] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:06:31.305089 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 666/1340] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:06:31 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:06:35.241385 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd537b-d68a-4d08-8912-477eed1b30f8 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:06:35.241884 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd537b-d68a-4d08-8912-477eed1b30f8 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:06:35.241990 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd537b-d68a-4d08-8912-477eed1b30f8 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:06:35.243298 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 675/1341] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:06:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:06:35.248773 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ae3f90-8445-4515-88b5-f9b44d325798 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:06:35.249183 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ae3f90-8445-4515-88b5-f9b44d325798 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:06:35.249437 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ae3f90-8445-4515-88b5-f9b44d325798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:06:35.314864 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-d0ae3f90-8445-4515-88b5-f9b44d325798 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:06:35.316083 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-d0ae3f90-8445-4515-88b5-f9b44d325798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:06:35.402690 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:06:35.414256 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 667/1342] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:06:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:06:35.697105 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bd537b-d68a-4d08-8912-477eed1b30f8 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:06:35.698830 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-f1bd537b-d68a-4d08-8912-477eed1b30f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:06:35.804673 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bd537b-d68a-4d08-8912-477eed1b30f8 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:06:35.844833 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:06:35.880062 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39664cc4-256d-44d8-ba84-a17121873660 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:06:35.880663 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39664cc4-256d-44d8-ba84-a17121873660 service neutron] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:06:35.880938 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39664cc4-256d-44d8-ba84-a17121873660 service neutron] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:06:35.881208 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39664cc4-256d-44d8-ba84-a17121873660 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:06:35.883677 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-39664cc4-256d-44d8-ba84-a17121873660 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:06:35.991816 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39664cc4-256d-44d8-ba84-a17121873660 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:06:35.993200 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39664cc4-256d-44d8-ba84-a17121873660 service neutron] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:06:36.011975 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 676/1343] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:06:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:06:38.507065 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f707cb9-5fe9-4955-99f2-ba76ec8130cb None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:06:38.508041 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f707cb9-5fe9-4955-99f2-ba76ec8130cb None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:06:38.508041 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f707cb9-5fe9-4955-99f2-ba76ec8130cb None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:06:38.509348 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 668/1344] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:06:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:06:38.515327 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d9f47f-f704-4d9a-b42a-bcf3c1b6e979 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:06:38.515842 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d9f47f-f704-4d9a-b42a-bcf3c1b6e979 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:06:38.515997 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d9f47f-f704-4d9a-b42a-bcf3c1b6e979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:06:38.589433 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-40d9f47f-f704-4d9a-b42a-bcf3c1b6e979 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:06:38.590651 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-40d9f47f-f704-4d9a-b42a-bcf3c1b6e979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:06:38.610346 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:06:38.693546 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 677/1345] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:06:38 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:06:38.974625 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f707cb9-5fe9-4955-99f2-ba76ec8130cb None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:06:38.976835 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-1f707cb9-5fe9-4955-99f2-ba76ec8130cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:06:39.079932 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f707cb9-5fe9-4955-99f2-ba76ec8130cb None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:06:39.154699 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b2f299-294d-43ea-be9d-9b8a373ad820 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:06:39.155340 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b2f299-294d-43ea-be9d-9b8a373ad820 service neutron] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:06:39.155615 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b2f299-294d-43ea-be9d-9b8a373ad820 service neutron] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:06:39.155883 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b2f299-294d-43ea-be9d-9b8a373ad820 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:06:39.157495 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-24b2f299-294d-43ea-be9d-9b8a373ad820 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:06:39.209882 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:06:39.280656 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b2f299-294d-43ea-be9d-9b8a373ad820 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:06:39.282022 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b2f299-294d-43ea-be9d-9b8a373ad820 service neutron] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:06:39.300918 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 669/1346] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:06:38 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:06:43.318501 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ddbbad-1624-4f46-9c90-c170c961fa1f None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:06:43.319066 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ddbbad-1624-4f46-9c90-c170c961fa1f None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:06:43.319189 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ddbbad-1624-4f46-9c90-c170c961fa1f None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:06:43.320599 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 678/1347] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:06:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:06:43.326007 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1a55e9-4746-4383-91e2-7cd748396263 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:06:43.326433 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1a55e9-4746-4383-91e2-7cd748396263 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:06:43.326629 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1a55e9-4746-4383-91e2-7cd748396263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:06:43.408162 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-3d1a55e9-4746-4383-91e2-7cd748396263 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:06:43.409314 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-3d1a55e9-4746-4383-91e2-7cd748396263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:06:43.513061 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:06:43.519520 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 670/1348] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:06:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:06:43.798158 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ddbbad-1624-4f46-9c90-c170c961fa1f None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:06:43.799765 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-59ddbbad-1624-4f46-9c90-c170c961fa1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:06:43.911380 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ddbbad-1624-4f46-9c90-c170c961fa1f None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:06:43.961360 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:06:43.988673 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0438e81b-e7b0-4ecc-b048-9d2cdae0ccd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:06:43.989277 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0438e81b-e7b0-4ecc-b048-9d2cdae0ccd0 service neutron] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:06:43.989552 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0438e81b-e7b0-4ecc-b048-9d2cdae0ccd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:06:43.989822 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0438e81b-e7b0-4ecc-b048-9d2cdae0ccd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:06:43.992019 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-0438e81b-e7b0-4ecc-b048-9d2cdae0ccd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:06:44.101552 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0438e81b-e7b0-4ecc-b048-9d2cdae0ccd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:06:44.103029 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0438e81b-e7b0-4ecc-b048-9d2cdae0ccd0 service neutron] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:06:44.121122 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 679/1349] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:06:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:06:49.039073 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f674601a-3d1a-45a9-b4f8-b49fdb7d2f38 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:06:49.039616 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f674601a-3d1a-45a9-b4f8-b49fdb7d2f38 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:06:49.040379 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f674601a-3d1a-45a9-b4f8-b49fdb7d2f38 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:06:49.041946 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 671/1350] 10.222.0.61 () {54 vars in 921 bytes} [Fri Aug 26 12:06:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:06:49.047918 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac42b6-1d0e-4260-a630-ccdc411f4076 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:06:49.047918 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac42b6-1d0e-4260-a630-ccdc411f4076 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:06:49.048426 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac42b6-1d0e-4260-a630-ccdc411f4076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:06:49.168521 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-5dac42b6-1d0e-4260-a630-ccdc411f4076 None None] MFA Rules not processed for user `7346e4a29dfb4a0a9768597704be286c`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:06:49.170334 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-5dac42b6-1d0e-4260-a630-ccdc411f4076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:06:49.195643 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:06:49.332331 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 680/1351] 10.222.0.61 () {58 vars in 1018 bytes} [Fri Aug 26 12:06:49 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:06:53.606931 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12bc8e-279f-4533-a051-e2118a4feb19 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:06:53.607864 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12bc8e-279f-4533-a051-e2118a4feb19 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:06:53.608127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12bc8e-279f-4533-a051-e2118a4feb19 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:06:53.609647 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 672/1352] 10.222.0.61 () {54 vars in 921 bytes} [Fri Aug 26 12:06:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:06:53.614617 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396b086-6872-4586-9927-f285c8f0e99a None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:06:53.615954 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396b086-6872-4586-9927-f285c8f0e99a None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:06:53.616411 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396b086-6872-4586-9927-f285c8f0e99a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:06:53.697212 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-a396b086-6872-4586-9927-f285c8f0e99a None None] MFA Rules not processed for user `7346e4a29dfb4a0a9768597704be286c`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:06:53.698536 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-a396b086-6872-4586-9927-f285c8f0e99a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:06:53.779425 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:06:53.820149 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 681/1353] 10.222.0.61 () {58 vars in 1018 bytes} [Fri Aug 26 12:06:53 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:06:55.061182 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4af6a7-d61d-4d86-b3d3-7f67fb60dd79 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:06:55.061708 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4af6a7-d61d-4d86-b3d3-7f67fb60dd79 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:06:55.061708 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4af6a7-d61d-4d86-b3d3-7f67fb60dd79 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:06:55.062952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 673/1354] 10.222.0.61 () {54 vars in 921 bytes} [Fri Aug 26 12:06:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:06:55.067982 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18624e31-71fe-4ee7-9985-19308b2b654f None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:06:55.068492 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18624e31-71fe-4ee7-9985-19308b2b654f None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:06:55.068706 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18624e31-71fe-4ee7-9985-19308b2b654f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:06:55.147502 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-18624e31-71fe-4ee7-9985-19308b2b654f None None] MFA Rules not processed for user `7346e4a29dfb4a0a9768597704be286c`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:06:55.148706 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-18624e31-71fe-4ee7-9985-19308b2b654f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:06:55.254428 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 682/1355] 10.222.0.61 () {58 vars in 1018 bytes} [Fri Aug 26 12:06:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:06:58.670783 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2898f256-ce1d-4a64-a66a-6f49009a151f None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:06:58.671364 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2898f256-ce1d-4a64-a66a-6f49009a151f None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:06:58.671364 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2898f256-ce1d-4a64-a66a-6f49009a151f None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:06:58.672718 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 674/1356] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:06:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:06:58.678087 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146be69d-d198-402e-a78a-c4f2a560b5d6 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:06:58.678479 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146be69d-d198-402e-a78a-c4f2a560b5d6 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:06:58.678769 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146be69d-d198-402e-a78a-c4f2a560b5d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:06:58.717476 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:06:58.759704 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-146be69d-d198-402e-a78a-c4f2a560b5d6 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:06:58.760884 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-146be69d-d198-402e-a78a-c4f2a560b5d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:06:58.882501 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 683/1357] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:06:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:06:59.188432 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609b42e4-b872-4539-a6e7-abd1c9b2f5f0 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:06:59.189035 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609b42e4-b872-4539-a6e7-abd1c9b2f5f0 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:06:59.190538 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609b42e4-b872-4539-a6e7-abd1c9b2f5f0 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:06:59.192239 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 675/1358] 10.222.0.61 () {54 vars in 962 bytes} [Fri Aug 26 12:06:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:06:59.201667 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fbfd82-4ba2-465f-b050-06ae2cb87ff2 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:06:59.201960 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fbfd82-4ba2-465f-b050-06ae2cb87ff2 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:06:59.202234 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fbfd82-4ba2-465f-b050-06ae2cb87ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:06:59.284390 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-c7fbfd82-4ba2-465f-b050-06ae2cb87ff2 None None] MFA Rules not processed for user `7aacd7d6d17e4f10ba5c9107a12beafa`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:06:59.285623 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-c7fbfd82-4ba2-465f-b050-06ae2cb87ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:06:59.340310 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:06:59.443519 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 684/1359] 10.222.0.61 () {58 vars in 1059 bytes} [Fri Aug 26 12:06:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:06:59.451071 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609b42e4-b872-4539-a6e7-abd1c9b2f5f0 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:06:59.452663 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-609b42e4-b872-4539-a6e7-abd1c9b2f5f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:06:59.634979 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609b42e4-b872-4539-a6e7-abd1c9b2f5f0 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:06:59.722621 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:06:59.778864 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d40fb81-6f39-46e0-98fc-027451f2342d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aacd7d6d17e4f10ba5c9107a12beafa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:06:59.779735 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d40fb81-6f39-46e0-98fc-027451f2342d service nova] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:06:59.780111 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d40fb81-6f39-46e0-98fc-027451f2342d service nova] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:06:59.780477 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d40fb81-6f39-46e0-98fc-027451f2342d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:06:59.782230 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-0d40fb81-6f39-46e0-98fc-027451f2342d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:06:59.884700 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d40fb81-6f39-46e0-98fc-027451f2342d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:06:59.886470 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d40fb81-6f39-46e0-98fc-027451f2342d service nova] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:06:59.906627 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 676/1360] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:06:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:07:02.079814 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5506be3-5a68-48e6-a4a4-d751f5909610 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:02.080391 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5506be3-5a68-48e6-a4a4-d751f5909610 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:02.080391 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5506be3-5a68-48e6-a4a4-d751f5909610 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:02.081842 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 685/1361] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:07:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:07:02.087877 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafba5bf-f686-402d-8535-8eaa54b6ffc4 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:02.088345 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafba5bf-f686-402d-8535-8eaa54b6ffc4 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:02.088504 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafba5bf-f686-402d-8535-8eaa54b6ffc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:02.158015 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-dafba5bf-f686-402d-8535-8eaa54b6ffc4 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:07:02.159515 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-dafba5bf-f686-402d-8535-8eaa54b6ffc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:02.200329 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:07:02.270245 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 677/1362] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:07:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:07:02.564837 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5506be3-5a68-48e6-a4a4-d751f5909610 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:07:02.566481 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-f5506be3-5a68-48e6-a4a4-d751f5909610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:02.723486 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5506be3-5a68-48e6-a4a4-d751f5909610 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:07:02.733496 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:07:02.855020 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8659fd1-9b43-423c-8547-58461248b484 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aacd7d6d17e4f10ba5c9107a12beafa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:07:02.855770 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8659fd1-9b43-423c-8547-58461248b484 service nova] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:02.856152 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8659fd1-9b43-423c-8547-58461248b484 service nova] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:02.856526 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8659fd1-9b43-423c-8547-58461248b484 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:02.858210 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-a8659fd1-9b43-423c-8547-58461248b484 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:02.969237 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8659fd1-9b43-423c-8547-58461248b484 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:07:02.971155 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8659fd1-9b43-423c-8547-58461248b484 service nova] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:07:02.995063 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 686/1363] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:07:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:07:05.249674 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8308be5f-9f0c-4eee-9a0e-42a2f0cd9803 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:05.250571 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8308be5f-9f0c-4eee-9a0e-42a2f0cd9803 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:05.250851 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8308be5f-9f0c-4eee-9a0e-42a2f0cd9803 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:05.252647 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 678/1364] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:07:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:07:05.258746 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5683cc03-8b9a-4ffa-a0f3-10ff1fd75cdc None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:05.259188 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5683cc03-8b9a-4ffa-a0f3-10ff1fd75cdc None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:05.259625 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5683cc03-8b9a-4ffa-a0f3-10ff1fd75cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:05.276988 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:07:05.331941 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-5683cc03-8b9a-4ffa-a0f3-10ff1fd75cdc None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:07:05.333786 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-5683cc03-8b9a-4ffa-a0f3-10ff1fd75cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:05.437843 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 687/1365] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:07:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:07:05.738390 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d715c9c8-72ed-4784-b2c3-471186e80801 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:05.738680 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d715c9c8-72ed-4784-b2c3-471186e80801 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:05.738957 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d715c9c8-72ed-4784-b2c3-471186e80801 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:05.741841 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 679/1366] 10.222.0.61 () {54 vars in 962 bytes} [Fri Aug 26 12:07:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:07:05.752198 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e880e0-a7f4-4c2e-ab85-2b3c15a265e8 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:05.752484 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e880e0-a7f4-4c2e-ab85-2b3c15a265e8 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:05.752756 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e880e0-a7f4-4c2e-ab85-2b3c15a265e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:05.833630 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-62e880e0-a7f4-4c2e-ab85-2b3c15a265e8 None None] MFA Rules not processed for user `7aacd7d6d17e4f10ba5c9107a12beafa`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:07:05.834968 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-62e880e0-a7f4-4c2e-ab85-2b3c15a265e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:05.874058 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:07:05.966498 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 688/1367] 10.222.0.61 () {58 vars in 1059 bytes} [Fri Aug 26 12:07:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:07:05.975998 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d715c9c8-72ed-4784-b2c3-471186e80801 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:07:05.977551 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-d715c9c8-72ed-4784-b2c3-471186e80801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:06.142130 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d715c9c8-72ed-4784-b2c3-471186e80801 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:07:06.168561 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:07:06.272068 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc9af2c-a976-4aff-b304-8d2dc7761ab5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aacd7d6d17e4f10ba5c9107a12beafa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:07:06.272743 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc9af2c-a976-4aff-b304-8d2dc7761ab5 service nova] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:06.273110 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc9af2c-a976-4aff-b304-8d2dc7761ab5 service nova] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:06.273482 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc9af2c-a976-4aff-b304-8d2dc7761ab5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:06.275191 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-6dc9af2c-a976-4aff-b304-8d2dc7761ab5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:06.372814 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc9af2c-a976-4aff-b304-8d2dc7761ab5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:07:06.374635 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc9af2c-a976-4aff-b304-8d2dc7761ab5 service nova] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:07:06.396625 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 680/1368] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:07:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:07:06.981763 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87feb7b-075a-49f5-aaf4-e83a22ac48bf None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:06.982278 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87feb7b-075a-49f5-aaf4-e83a22ac48bf None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:06.982278 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87feb7b-075a-49f5-aaf4-e83a22ac48bf None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:06.983736 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 689/1369] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:07:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:07:06.988894 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d574b65f-819f-4a1f-b13a-6fee7e26903e None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:06.989748 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d574b65f-819f-4a1f-b13a-6fee7e26903e None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:06.990021 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d574b65f-819f-4a1f-b13a-6fee7e26903e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:07.025282 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:07:07.061067 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-d574b65f-819f-4a1f-b13a-6fee7e26903e None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:07:07.062278 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-d574b65f-819f-4a1f-b13a-6fee7e26903e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:07.163134 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 681/1370] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:07:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:07:07.718049 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f85e70-279f-4410-a22b-9d523f896564 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:07.718528 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f85e70-279f-4410-a22b-9d523f896564 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:07.718972 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f85e70-279f-4410-a22b-9d523f896564 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:07.720814 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 690/1371] 10.222.0.61 () {54 vars in 964 bytes} [Fri Aug 26 12:07:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:07:07.726001 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e0216a-b744-4309-9a95-d55848c9c990 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:07.726430 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e0216a-b744-4309-9a95-d55848c9c990 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:07.726734 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e0216a-b744-4309-9a95-d55848c9c990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:07.810345 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-b7e0216a-b744-4309-9a95-d55848c9c990 None None] MFA Rules not processed for user `51239ac314e245e8a5706e65b133db3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:07:07.812748 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-b7e0216a-b744-4309-9a95-d55848c9c990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:07.863728 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:07:07.897268 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 682/1372] 10.222.0.61 () {58 vars in 1061 bytes} [Fri Aug 26 12:07:07 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:07:07.904645 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f85e70-279f-4410-a22b-9d523f896564 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:07:07.906161 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-a4f85e70-279f-4410-a22b-9d523f896564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:08.013610 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f85e70-279f-4410-a22b-9d523f896564 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:07:08.060674 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:07:08.099670 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b0b8e5-db85-4cd5-9f40-ec1154ad0056 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:07:08.100290 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b0b8e5-db85-4cd5-9f40-ec1154ad0056 service cinder] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:08.100566 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b0b8e5-db85-4cd5-9f40-ec1154ad0056 service cinder] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:08.100833 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b0b8e5-db85-4cd5-9f40-ec1154ad0056 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:08.102488 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-45b0b8e5-db85-4cd5-9f40-ec1154ad0056 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:08.233192 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b0b8e5-db85-4cd5-9f40-ec1154ad0056 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:07:08.235021 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b0b8e5-db85-4cd5-9f40-ec1154ad0056 service cinder] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:07:08.258850 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 691/1373] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:07:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:07:08.388453 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163aba47-c19c-4d08-87e3-1fc153887139 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:08.388744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163aba47-c19c-4d08-87e3-1fc153887139 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:08.389023 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163aba47-c19c-4d08-87e3-1fc153887139 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:08.390378 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 683/1374] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:07:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:07:08.396160 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25afc43-2189-4f7e-9e9d-b55d187d4d80 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:08.396516 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25afc43-2189-4f7e-9e9d-b55d187d4d80 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:08.396709 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25afc43-2189-4f7e-9e9d-b55d187d4d80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:08.464568 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-b25afc43-2189-4f7e-9e9d-b55d187d4d80 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:07:08.465683 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-b25afc43-2189-4f7e-9e9d-b55d187d4d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:08.488473 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:07:08.571367 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 692/1375] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:07:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:07:08.855898 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163aba47-c19c-4d08-87e3-1fc153887139 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:07:08.857410 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-163aba47-c19c-4d08-87e3-1fc153887139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:09.022816 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163aba47-c19c-4d08-87e3-1fc153887139 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:07:09.075887 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:07:09.137619 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e19c4a3-361f-4ae9-aa10-bf27decb96ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aacd7d6d17e4f10ba5c9107a12beafa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:07:09.138220 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e19c4a3-361f-4ae9-aa10-bf27decb96ec service nova] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:09.138495 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e19c4a3-361f-4ae9-aa10-bf27decb96ec service nova] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:09.138761 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e19c4a3-361f-4ae9-aa10-bf27decb96ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:09.140404 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-5e19c4a3-361f-4ae9-aa10-bf27decb96ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:09.229600 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e19c4a3-361f-4ae9-aa10-bf27decb96ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:07:09.231226 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e19c4a3-361f-4ae9-aa10-bf27decb96ec service nova] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:07:09.249158 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 684/1376] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:07:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:07:11.215080 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fe38c0-caa8-40e5-bafd-b5b1233a667e None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:11.215638 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fe38c0-caa8-40e5-bafd-b5b1233a667e None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:11.215638 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fe38c0-caa8-40e5-bafd-b5b1233a667e None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:11.217077 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 693/1377] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:07:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:07:11.222231 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7767ea-2b7a-4825-bd55-acf38f8a77ba None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:11.223000 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7767ea-2b7a-4825-bd55-acf38f8a77ba None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:11.223865 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7767ea-2b7a-4825-bd55-acf38f8a77ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:11.296537 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-df7767ea-2b7a-4825-bd55-acf38f8a77ba None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:07:11.297938 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-df7767ea-2b7a-4825-bd55-acf38f8a77ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:11.308266 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:07:11.416491 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 685/1378] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:07:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:07:11.715041 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fe38c0-caa8-40e5-bafd-b5b1233a667e None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:07:11.717307 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-19fe38c0-caa8-40e5-bafd-b5b1233a667e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:11.932593 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fe38c0-caa8-40e5-bafd-b5b1233a667e None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:07:11.936880 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:07:12.057893 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9a1787-0a6d-4dd1-b69e-2a227fa87018 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aacd7d6d17e4f10ba5c9107a12beafa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:07:12.058411 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a1787-0a6d-4dd1-b69e-2a227fa87018 service nova] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:12.058843 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a1787-0a6d-4dd1-b69e-2a227fa87018 service nova] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:12.059121 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a1787-0a6d-4dd1-b69e-2a227fa87018 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:12.060831 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-0b9a1787-0a6d-4dd1-b69e-2a227fa87018 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:12.158957 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9a1787-0a6d-4dd1-b69e-2a227fa87018 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:07:12.160963 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9a1787-0a6d-4dd1-b69e-2a227fa87018 service nova] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:07:12.181639 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 694/1379] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:07:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:07:14.127009 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97259b4-45ee-49cd-a0a7-1f69e4a31d67 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:14.127538 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97259b4-45ee-49cd-a0a7-1f69e4a31d67 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:14.127617 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97259b4-45ee-49cd-a0a7-1f69e4a31d67 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:14.129132 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 686/1380] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:07:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:07:14.134453 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a05de2-eace-41d3-a3e1-8caf81c0c608 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:14.134940 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a05de2-eace-41d3-a3e1-8caf81c0c608 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:14.135032 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a05de2-eace-41d3-a3e1-8caf81c0c608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:14.146713 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:07:14.209946 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-39a05de2-eace-41d3-a3e1-8caf81c0c608 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:07:14.211146 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-39a05de2-eace-41d3-a3e1-8caf81c0c608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:14.314828 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 695/1381] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:07:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:07:14.604561 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97259b4-45ee-49cd-a0a7-1f69e4a31d67 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:07:14.606081 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-a97259b4-45ee-49cd-a0a7-1f69e4a31d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:14.767003 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:07:14.779722 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97259b4-45ee-49cd-a0a7-1f69e4a31d67 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:07:14.901693 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b079da9-5870-41eb-91d3-30e338d32d6a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aacd7d6d17e4f10ba5c9107a12beafa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:07:14.902295 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b079da9-5870-41eb-91d3-30e338d32d6a service nova] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:14.902571 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b079da9-5870-41eb-91d3-30e338d32d6a service nova] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:14.902837 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b079da9-5870-41eb-91d3-30e338d32d6a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:14.904481 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-4b079da9-5870-41eb-91d3-30e338d32d6a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:14.997510 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b079da9-5870-41eb-91d3-30e338d32d6a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:07:14.999240 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b079da9-5870-41eb-91d3-30e338d32d6a service nova] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:07:15.010102 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:07:15.019037 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 687/1382] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:07:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:07:17.295700 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f8d0c6-44d0-48d2-be54-ac66e58db0ce None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:17.296562 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f8d0c6-44d0-48d2-be54-ac66e58db0ce None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:17.296562 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f8d0c6-44d0-48d2-be54-ac66e58db0ce None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:17.297689 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 696/1383] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:07:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:07:17.303907 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3cd429-6e29-48cf-bc08-c6591a779b4c None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:17.304466 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3cd429-6e29-48cf-bc08-c6591a779b4c None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:17.304466 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3cd429-6e29-48cf-bc08-c6591a779b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:17.373211 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-5c3cd429-6e29-48cf-bc08-c6591a779b4c None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:07:17.374456 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-5c3cd429-6e29-48cf-bc08-c6591a779b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:17.471886 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 688/1384] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:07:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:07:17.762534 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f8d0c6-44d0-48d2-be54-ac66e58db0ce None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:07:17.764526 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-26f8d0c6-44d0-48d2-be54-ac66e58db0ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:17.864038 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:07:17.928737 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f8d0c6-44d0-48d2-be54-ac66e58db0ce None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:07:18.046004 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035b514a-a5e2-456d-a35e-1d9814c57979 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aacd7d6d17e4f10ba5c9107a12beafa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:07:18.046556 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035b514a-a5e2-456d-a35e-1d9814c57979 service nova] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:18.046829 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035b514a-a5e2-456d-a35e-1d9814c57979 service nova] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:18.047130 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035b514a-a5e2-456d-a35e-1d9814c57979 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:18.049645 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-035b514a-a5e2-456d-a35e-1d9814c57979 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:18.107638 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:07:18.149993 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035b514a-a5e2-456d-a35e-1d9814c57979 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:07:18.151630 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035b514a-a5e2-456d-a35e-1d9814c57979 service nova] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:07:18.170916 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 697/1385] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:07:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:07:20.136216 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f4b2a7-5274-47df-a37e-dda361ed786a None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:20.136779 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f4b2a7-5274-47df-a37e-dda361ed786a None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:20.136779 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f4b2a7-5274-47df-a37e-dda361ed786a None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:20.138123 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 689/1386] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:07:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:07:20.143512 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcac567-af01-48a4-9674-2954ee5ca5a8 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:20.144116 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcac567-af01-48a4-9674-2954ee5ca5a8 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:20.144253 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcac567-af01-48a4-9674-2954ee5ca5a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:20.212661 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-0dcac567-af01-48a4-9674-2954ee5ca5a8 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:07:20.213933 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-0dcac567-af01-48a4-9674-2954ee5ca5a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:20.322237 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 698/1387] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:07:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:07:20.611990 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f4b2a7-5274-47df-a37e-dda361ed786a None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:07:20.613538 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-85f4b2a7-5274-47df-a37e-dda361ed786a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:20.695848 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:07:20.772281 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f4b2a7-5274-47df-a37e-dda361ed786a None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:07:20.883322 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cb7b60-a6e8-4174-be29-7152846ce6e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aacd7d6d17e4f10ba5c9107a12beafa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:07:20.883930 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cb7b60-a6e8-4174-be29-7152846ce6e8 service nova] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:20.884204 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cb7b60-a6e8-4174-be29-7152846ce6e8 service nova] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:20.884473 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cb7b60-a6e8-4174-be29-7152846ce6e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:20.886063 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-68cb7b60-a6e8-4174-be29-7152846ce6e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:20.924969 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:07:20.985070 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cb7b60-a6e8-4174-be29-7152846ce6e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:07:20.987011 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cb7b60-a6e8-4174-be29-7152846ce6e8 service nova] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:07:21.008764 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 690/1388] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:07:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:07:22.987221 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6604c4-8c1c-46f7-a4b7-5cdd3ce9bd07 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:22.987725 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6604c4-8c1c-46f7-a4b7-5cdd3ce9bd07 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:22.987725 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6604c4-8c1c-46f7-a4b7-5cdd3ce9bd07 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:22.989087 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 699/1389] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:07:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:07:22.994337 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c846a-fa1c-4680-b83f-79f561556e97 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:22.994803 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c846a-fa1c-4680-b83f-79f561556e97 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:22.994905 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c846a-fa1c-4680-b83f-79f561556e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:23.059646 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-ae9c846a-fa1c-4680-b83f-79f561556e97 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:07:23.060843 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-ae9c846a-fa1c-4680-b83f-79f561556e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:23.158151 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 691/1390] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:07:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:07:23.442959 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6604c4-8c1c-46f7-a4b7-5cdd3ce9bd07 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:07:23.444508 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-5b6604c4-8c1c-46f7-a4b7-5cdd3ce9bd07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:23.475059 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:07:23.590499 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6604c4-8c1c-46f7-a4b7-5cdd3ce9bd07 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:07:23.692071 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:07:23.704040 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f30f365-0a87-46a5-b093-ce31c86fe48c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aacd7d6d17e4f10ba5c9107a12beafa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:07:23.704644 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f30f365-0a87-46a5-b093-ce31c86fe48c service nova] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:23.704922 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f30f365-0a87-46a5-b093-ce31c86fe48c service nova] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:23.705191 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f30f365-0a87-46a5-b093-ce31c86fe48c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:23.707329 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-8f30f365-0a87-46a5-b093-ce31c86fe48c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:23.804967 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f30f365-0a87-46a5-b093-ce31c86fe48c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:07:23.806556 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f30f365-0a87-46a5-b093-ce31c86fe48c service nova] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:07:23.824839 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 700/1391] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:07:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:07:24.725977 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5905ee95-ce33-48ae-b4d3-8ca4f3f93404 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:24.726456 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5905ee95-ce33-48ae-b4d3-8ca4f3f93404 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:24.726552 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5905ee95-ce33-48ae-b4d3-8ca4f3f93404 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:24.727938 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 692/1392] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:07:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:07:24.733273 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7441b8-1854-4cda-a3ad-488e13b92f7e None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:24.733726 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7441b8-1854-4cda-a3ad-488e13b92f7e None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:24.733828 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7441b8-1854-4cda-a3ad-488e13b92f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:24.800057 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-8b7441b8-1854-4cda-a3ad-488e13b92f7e None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:07:24.801409 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-8b7441b8-1854-4cda-a3ad-488e13b92f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:24.866492 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:07:24.909263 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 701/1393] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:07:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:07:25.215555 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9970de7-9d59-433d-94af-fc22909613c8 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:25.215844 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9970de7-9d59-433d-94af-fc22909613c8 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:25.216128 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9970de7-9d59-433d-94af-fc22909613c8 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:25.217467 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 693/1394] 10.222.0.61 () {54 vars in 967 bytes} [Fri Aug 26 12:07:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:07:25.223082 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94c56d2-c1f7-4603-9183-153e6ec3e8d5 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:25.223409 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94c56d2-c1f7-4603-9183-153e6ec3e8d5 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:25.223686 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94c56d2-c1f7-4603-9183-153e6ec3e8d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:25.299777 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-b94c56d2-c1f7-4603-9183-153e6ec3e8d5 None None] MFA Rules not processed for user `152fd7c708114f249b9c960165f0fdfe`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:07:25.300997 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-b94c56d2-c1f7-4603-9183-153e6ec3e8d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:25.377441 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 702/1395] 10.222.0.61 () {58 vars in 1064 bytes} [Fri Aug 26 12:07:25 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:07:25.385996 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9970de7-9d59-433d-94af-fc22909613c8 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:07:25.387545 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-b9970de7-9d59-433d-94af-fc22909613c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:25.401639 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:07:25.490900 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9970de7-9d59-433d-94af-fc22909613c8 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:07:25.561903 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841abdf5-f6a8-4206-b711-dafaeaf983b6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152fd7c708114f249b9c960165f0fdfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:07:25.562503 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841abdf5-f6a8-4206-b711-dafaeaf983b6 service glance] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:25.562778 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841abdf5-f6a8-4206-b711-dafaeaf983b6 service glance] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:25.563046 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841abdf5-f6a8-4206-b711-dafaeaf983b6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:25.564667 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-841abdf5-f6a8-4206-b711-dafaeaf983b6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:25.637890 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:07:25.679371 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841abdf5-f6a8-4206-b711-dafaeaf983b6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:07:25.680801 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841abdf5-f6a8-4206-b711-dafaeaf983b6 service glance] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:07:25.699800 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 694/1396] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:07:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:07:25.747860 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c615a5-fbb1-4970-bc09-b2e7090916b3 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:25.748319 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c615a5-fbb1-4970-bc09-b2e7090916b3 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:25.748434 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c615a5-fbb1-4970-bc09-b2e7090916b3 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:25.749783 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 703/1397] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:07:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:07:25.755522 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c71a91c-8e0d-4045-964d-36d871dd9e64 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:25.755933 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c71a91c-8e0d-4045-964d-36d871dd9e64 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:25.756090 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c71a91c-8e0d-4045-964d-36d871dd9e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:25.821670 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-3c71a91c-8e0d-4045-964d-36d871dd9e64 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:07:25.822846 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-3c71a91c-8e0d-4045-964d-36d871dd9e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:25.921935 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 695/1398] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:07:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:07:26.213476 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c615a5-fbb1-4970-bc09-b2e7090916b3 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:07:26.215054 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-33c615a5-fbb1-4970-bc09-b2e7090916b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:26.279842 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:07:26.377864 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c615a5-fbb1-4970-bc09-b2e7090916b3 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:07:26.437241 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3897f73-7009-4c45-a0fb-1fc49f0fdeca None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:26.437462 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3897f73-7009-4c45-a0fb-1fc49f0fdeca None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:26.437742 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3897f73-7009-4c45-a0fb-1fc49f0fdeca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:26.464101 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:07:26.498960 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5dec51-dc7f-4943-882f-cf331138e2a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aacd7d6d17e4f10ba5c9107a12beafa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:07:26.499599 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5dec51-dc7f-4943-882f-cf331138e2a8 service nova] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:26.499873 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5dec51-dc7f-4943-882f-cf331138e2a8 service nova] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:26.500137 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5dec51-dc7f-4943-882f-cf331138e2a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:26.501760 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-0c5dec51-dc7f-4943-882f-cf331138e2a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:26.509864 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-f3897f73-7009-4c45-a0fb-1fc49f0fdeca None None] MFA Rules not processed for user `755b04cae45746569bfbf92f2f213a58`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:07:26.511109 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-f3897f73-7009-4c45-a0fb-1fc49f0fdeca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:26.524183 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:07:26.594873 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5dec51-dc7f-4943-882f-cf331138e2a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:07:26.596519 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5dec51-dc7f-4943-882f-cf331138e2a8 service nova] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:07:26.601043 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 696/1399] 10.222.0.61 () {58 vars in 1009 bytes} [Fri Aug 26 12:07:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:07:26.615935 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 704/1400] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:07:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:07:26.650124 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe67b3e-ed2e-4375-a44b-ad43ef3c2adb None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:26.650489 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe67b3e-ed2e-4375-a44b-ad43ef3c2adb None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:26.650835 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe67b3e-ed2e-4375-a44b-ad43ef3c2adb None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:26.652283 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 697/1401] 10.222.0.61 () {54 vars in 943 bytes} [Fri Aug 26 12:07:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:07:26.658857 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a2ebc9-b3a2-411b-83d5-6f3820a1eea7 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:26.659145 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a2ebc9-b3a2-411b-83d5-6f3820a1eea7 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:26.659644 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a2ebc9-b3a2-411b-83d5-6f3820a1eea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:26.750525 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-27a2ebc9-b3a2-411b-83d5-6f3820a1eea7 None None] MFA Rules not processed for user `ead95a244d11432d9fe9f718e1a75f38`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:07:26.751803 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-27a2ebc9-b3a2-411b-83d5-6f3820a1eea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:26.837854 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:07:26.867260 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 705/1402] 10.222.0.61 () {58 vars in 1040 bytes} [Fri Aug 26 12:07:26 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:07:26.876663 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe67b3e-ed2e-4375-a44b-ad43ef3c2adb None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:07:26.878191 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-8fe67b3e-ed2e-4375-a44b-ad43ef3c2adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:26.985255 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:07:27.014191 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe67b3e-ed2e-4375-a44b-ad43ef3c2adb None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:07:27.119405 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff52c51-ace2-4e48-8cce-f1a50d7e8caf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead95a244d11432d9fe9f718e1a75f38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:07:27.120020 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff52c51-ace2-4e48-8cce-f1a50d7e8caf service swift] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:27.120296 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff52c51-ace2-4e48-8cce-f1a50d7e8caf service swift] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:27.120561 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff52c51-ace2-4e48-8cce-f1a50d7e8caf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:27.122180 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-6ff52c51-ace2-4e48-8cce-f1a50d7e8caf service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:27.207643 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff52c51-ace2-4e48-8cce-f1a50d7e8caf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:07:27.209236 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff52c51-ace2-4e48-8cce-f1a50d7e8caf service swift] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:07:27.217145 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 698/1403] 10.222.0.61 () {60 vars in 1384 bytes} [Fri Aug 26 12:07:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 12:07:28.590846 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba1817-d63f-45db-920a-a37ea5683063 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:28.591460 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba1817-d63f-45db-920a-a37ea5683063 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:28.591460 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba1817-d63f-45db-920a-a37ea5683063 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:28.596607 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 706/1404] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:07:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:07:28.598691 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f6083f-637b-4437-8f22-5e0b67fcb7cb None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:28.599111 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f6083f-637b-4437-8f22-5e0b67fcb7cb None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:28.599527 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f6083f-637b-4437-8f22-5e0b67fcb7cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:28.616911 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:07:28.672459 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-b4f6083f-637b-4437-8f22-5e0b67fcb7cb None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:07:28.673685 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-b4f6083f-637b-4437-8f22-5e0b67fcb7cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:28.795743 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 699/1405] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:07:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:07:29.088553 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba1817-d63f-45db-920a-a37ea5683063 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:07:29.090069 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-aaba1817-d63f-45db-920a-a37ea5683063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:29.274413 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba1817-d63f-45db-920a-a37ea5683063 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:07:29.336764 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:07:29.408744 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f09505-d939-47bf-9559-8f270c39d929 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aacd7d6d17e4f10ba5c9107a12beafa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:07:29.409543 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f09505-d939-47bf-9559-8f270c39d929 service nova] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:29.409959 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f09505-d939-47bf-9559-8f270c39d929 service nova] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:29.410237 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f09505-d939-47bf-9559-8f270c39d929 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:29.411955 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-d5f09505-d939-47bf-9559-8f270c39d929 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:29.523801 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f09505-d939-47bf-9559-8f270c39d929 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:07:29.525161 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f09505-d939-47bf-9559-8f270c39d929 service nova] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:07:29.551445 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 707/1406] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:07:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:07:31.526247 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528d41c5-6af9-45b3-af16-1e5e8947fe09 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:31.526826 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528d41c5-6af9-45b3-af16-1e5e8947fe09 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:31.526826 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528d41c5-6af9-45b3-af16-1e5e8947fe09 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:31.531343 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 700/1407] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:07:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:07:31.533904 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c6e6d2-cf92-4408-ab8d-1d55fb754009 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:31.534903 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c6e6d2-cf92-4408-ab8d-1d55fb754009 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:31.535059 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c6e6d2-cf92-4408-ab8d-1d55fb754009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:31.607683 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-c7c6e6d2-cf92-4408-ab8d-1d55fb754009 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:07:31.608992 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-c7c6e6d2-cf92-4408-ab8d-1d55fb754009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:31.625214 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:07:31.717412 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 708/1408] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:07:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:07:32.022375 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528d41c5-6af9-45b3-af16-1e5e8947fe09 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:07:32.024038 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-528d41c5-6af9-45b3-af16-1e5e8947fe09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:32.121259 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:07:32.189312 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528d41c5-6af9-45b3-af16-1e5e8947fe09 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:07:32.314596 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562ebba4-59e9-4338-a52d-e09dfaebb276 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aacd7d6d17e4f10ba5c9107a12beafa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:07:32.315123 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ebba4-59e9-4338-a52d-e09dfaebb276 service nova] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:32.316299 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ebba4-59e9-4338-a52d-e09dfaebb276 service nova] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:32.317062 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ebba4-59e9-4338-a52d-e09dfaebb276 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:32.318699 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-562ebba4-59e9-4338-a52d-e09dfaebb276 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:32.384422 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:07:32.423744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562ebba4-59e9-4338-a52d-e09dfaebb276 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:07:32.425941 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562ebba4-59e9-4338-a52d-e09dfaebb276 service nova] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:07:32.445602 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 701/1409] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:07:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:07:41.716306 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=37 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:07:42.442216 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=22 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:07:49.142267 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeffe14-82c7-4104-8eed-767553906fe2 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:49.142901 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeffe14-82c7-4104-8eed-767553906fe2 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:49.142901 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeffe14-82c7-4104-8eed-767553906fe2 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:49.144225 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 709/1410] 10.222.0.61 () {54 vars in 918 bytes} [Fri Aug 26 12:07:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:07:49.149195 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c7b4b-c801-4bf1-b2c3-35a89e3c87b6 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:49.149599 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c7b4b-c801-4bf1-b2c3-35a89e3c87b6 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:49.149750 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c7b4b-c801-4bf1-b2c3-35a89e3c87b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:49.223462 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-568c7b4b-c801-4bf1-b2c3-35a89e3c87b6 None None] MFA Rules not processed for user `7346e4a29dfb4a0a9768597704be286c`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:07:49.224681 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-568c7b4b-c801-4bf1-b2c3-35a89e3c87b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:49.328595 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 702/1411] 10.222.0.61 () {58 vars in 1015 bytes} [Fri Aug 26 12:07:49 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:07:49.342848 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3554714e-6313-4c91-b1d9-85097419d3a3 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:49.343131 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3554714e-6313-4c91-b1d9-85097419d3a3 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:49.343443 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3554714e-6313-4c91-b1d9-85097419d3a3 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:49.344804 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 710/1412] 10.222.0.61 () {54 vars in 918 bytes} [Fri Aug 26 12:07:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:07:49.349953 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa2d6a-890c-4a96-94da-9a22f825d94f None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:49.350243 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa2d6a-890c-4a96-94da-9a22f825d94f None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:49.350523 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa2d6a-890c-4a96-94da-9a22f825d94f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:49.423549 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:07:49.427415 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-d0fa2d6a-890c-4a96-94da-9a22f825d94f None None] MFA Rules not processed for user `7aacd7d6d17e4f10ba5c9107a12beafa`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:07:49.428968 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-d0fa2d6a-890c-4a96-94da-9a22f825d94f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:49.542774 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 703/1413] 10.222.0.61 () {58 vars in 1015 bytes} [Fri Aug 26 12:07:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:07:51.923034 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b662758-e9c9-4a78-8b8f-ad5119902901 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:51.923712 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b662758-e9c9-4a78-8b8f-ad5119902901 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:51.923712 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b662758-e9c9-4a78-8b8f-ad5119902901 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:51.925131 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 711/1414] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 12:07:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:07:51.930479 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb52fd-95fe-4a50-9be8-428154c39e22 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:51.930935 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb52fd-95fe-4a50-9be8-428154c39e22 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:51.931047 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb52fd-95fe-4a50-9be8-428154c39e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:51.996106 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-7bcb52fd-95fe-4a50-9be8-428154c39e22 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:07:51.997199 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-7bcb52fd-95fe-4a50-9be8-428154c39e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:52.050824 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:07:52.091865 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 704/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 12:07:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:07:52.384401 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b662758-e9c9-4a78-8b8f-ad5119902901 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:07:52.385998 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-5b662758-e9c9-4a78-8b8f-ad5119902901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:52.532685 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b662758-e9c9-4a78-8b8f-ad5119902901 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:07:52.604142 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:07:52.646428 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adb462a-6c52-48b2-8ad8-4b6adb0a1a35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aacd7d6d17e4f10ba5c9107a12beafa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:07:52.646951 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb462a-6c52-48b2-8ad8-4b6adb0a1a35 service nova] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:52.647251 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb462a-6c52-48b2-8ad8-4b6adb0a1a35 service nova] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:52.647526 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb462a-6c52-48b2-8ad8-4b6adb0a1a35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:52.649128 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-5adb462a-6c52-48b2-8ad8-4b6adb0a1a35 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:52.738014 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adb462a-6c52-48b2-8ad8-4b6adb0a1a35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:07:52.739668 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adb462a-6c52-48b2-8ad8-4b6adb0a1a35 service nova] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:07:52.759057 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 712/1416] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:07:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:07:53.650403 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf9dc95-655a-4a02-b9a1-23f4f30e53c0 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:53.650973 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf9dc95-655a-4a02-b9a1-23f4f30e53c0 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:53.650973 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf9dc95-655a-4a02-b9a1-23f4f30e53c0 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:53.652321 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 705/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 12:07:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:07:53.657309 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df8d7d-5115-4f82-a9d3-ef846427d569 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:53.657733 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df8d7d-5115-4f82-a9d3-ef846427d569 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:53.658509 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df8d7d-5115-4f82-a9d3-ef846427d569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:53.733813 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-83df8d7d-5115-4f82-a9d3-ef846427d569 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:07:53.735012 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-83df8d7d-5115-4f82-a9d3-ef846427d569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:53.764583 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:07:53.834203 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 713/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 12:07:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:07:53.992898 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf9dc95-655a-4a02-b9a1-23f4f30e53c0 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:07:53.994387 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-8cf9dc95-655a-4a02-b9a1-23f4f30e53c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:54.138335 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf9dc95-655a-4a02-b9a1-23f4f30e53c0 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:07:54.179484 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:07:54.250997 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed942aa-89da-4220-84b9-6874aeb4a322 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aacd7d6d17e4f10ba5c9107a12beafa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:07:54.252159 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed942aa-89da-4220-84b9-6874aeb4a322 service nova] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:54.252436 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed942aa-89da-4220-84b9-6874aeb4a322 service nova] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:54.252701 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed942aa-89da-4220-84b9-6874aeb4a322 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:54.254594 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-8ed942aa-89da-4220-84b9-6874aeb4a322 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:54.340109 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed942aa-89da-4220-84b9-6874aeb4a322 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:07:54.341690 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed942aa-89da-4220-84b9-6874aeb4a322 service nova] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:07:54.359130 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 706/1419] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:07:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:07:55.239895 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89abf9ca-2672-482b-8f3c-13b8fba44f76 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:55.240432 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89abf9ca-2672-482b-8f3c-13b8fba44f76 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:55.240432 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89abf9ca-2672-482b-8f3c-13b8fba44f76 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:55.241807 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 714/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 12:07:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:07:55.246719 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c900c08a-811d-41eb-9f21-47caf571545c None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:55.247142 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c900c08a-811d-41eb-9f21-47caf571545c None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:55.247296 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c900c08a-811d-41eb-9f21-47caf571545c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:55.302031 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:07:55.324632 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-c900c08a-811d-41eb-9f21-47caf571545c None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:07:55.325841 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-c900c08a-811d-41eb-9f21-47caf571545c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:55.420954 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 707/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 12:07:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:07:55.570157 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89abf9ca-2672-482b-8f3c-13b8fba44f76 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:07:55.571813 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-89abf9ca-2672-482b-8f3c-13b8fba44f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:55.715455 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89abf9ca-2672-482b-8f3c-13b8fba44f76 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:07:55.727195 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:07:55.828975 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b4ad9b-944b-4730-814d-5dbd2d39ad73 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aacd7d6d17e4f10ba5c9107a12beafa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:07:55.830173 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b4ad9b-944b-4730-814d-5dbd2d39ad73 service nova] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:55.830559 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b4ad9b-944b-4730-814d-5dbd2d39ad73 service nova] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:55.830834 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b4ad9b-944b-4730-814d-5dbd2d39ad73 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:55.832457 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-54b4ad9b-944b-4730-814d-5dbd2d39ad73 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:55.919719 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b4ad9b-944b-4730-814d-5dbd2d39ad73 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:07:55.921284 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b4ad9b-944b-4730-814d-5dbd2d39ad73 service nova] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:07:55.939141 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 715/1422] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:07:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:07:56.801255 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb9169d-9046-4c32-848f-95b8e91131cb None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:56.801749 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb9169d-9046-4c32-848f-95b8e91131cb None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:56.801862 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb9169d-9046-4c32-848f-95b8e91131cb None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:56.803155 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 708/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 12:07:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:07:56.808171 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4509afe6-d877-49f5-a4c6-969a1a986039 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:56.808605 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4509afe6-d877-49f5-a4c6-969a1a986039 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:56.808703 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4509afe6-d877-49f5-a4c6-969a1a986039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:56.828611 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:07:56.884579 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-4509afe6-d877-49f5-a4c6-969a1a986039 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:07:56.885773 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-4509afe6-d877-49f5-a4c6-969a1a986039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:56.980683 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 716/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 12:07:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:07:57.131068 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb9169d-9046-4c32-848f-95b8e91131cb None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:07:57.132611 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-4fb9169d-9046-4c32-848f-95b8e91131cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:57.240920 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:07:57.277069 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb9169d-9046-4c32-848f-95b8e91131cb None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:07:57.386962 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07886443-135e-43f1-8869-51a12d8d6fa2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aacd7d6d17e4f10ba5c9107a12beafa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:07:57.387588 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07886443-135e-43f1-8869-51a12d8d6fa2 service nova] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:57.387868 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07886443-135e-43f1-8869-51a12d8d6fa2 service nova] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:57.388145 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07886443-135e-43f1-8869-51a12d8d6fa2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:57.389742 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-07886443-135e-43f1-8869-51a12d8d6fa2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:57.464304 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:07:57.478400 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07886443-135e-43f1-8869-51a12d8d6fa2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:07:57.480017 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07886443-135e-43f1-8869-51a12d8d6fa2 service nova] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:07:57.498172 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 709/1425] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:07:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:07:58.352969 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843d4bba-0031-4ba4-bf23-91a0920d53b2 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:58.353497 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843d4bba-0031-4ba4-bf23-91a0920d53b2 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:58.353497 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843d4bba-0031-4ba4-bf23-91a0920d53b2 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:58.354788 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 717/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 12:07:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:07:58.359775 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e28ea2-8def-4512-8812-8f8bd4dbf2f4 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:58.360155 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e28ea2-8def-4512-8812-8f8bd4dbf2f4 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:58.360329 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e28ea2-8def-4512-8812-8f8bd4dbf2f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:58.434740 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-a5e28ea2-8def-4512-8812-8f8bd4dbf2f4 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:07:58.435964 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-a5e28ea2-8def-4512-8812-8f8bd4dbf2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:58.530664 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 710/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 12:07:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:07:58.681816 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843d4bba-0031-4ba4-bf23-91a0920d53b2 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:07:58.683384 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-843d4bba-0031-4ba4-bf23-91a0920d53b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:58.771414 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:07:58.832708 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843d4bba-0031-4ba4-bf23-91a0920d53b2 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:07:58.945702 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c356d58b-151e-4098-ac31-e57f4d10bfd1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aacd7d6d17e4f10ba5c9107a12beafa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:07:58.946238 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c356d58b-151e-4098-ac31-e57f4d10bfd1 service nova] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:07:58.946519 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c356d58b-151e-4098-ac31-e57f4d10bfd1 service nova] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:07:58.946793 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c356d58b-151e-4098-ac31-e57f4d10bfd1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:07:58.948484 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-c356d58b-151e-4098-ac31-e57f4d10bfd1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:07:58.996088 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:07:59.037291 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c356d58b-151e-4098-ac31-e57f4d10bfd1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:07:59.038874 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c356d58b-151e-4098-ac31-e57f4d10bfd1 service nova] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:07:59.056375 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 718/1428] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:07:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:08:00.009257 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5309c87c-3ae9-4999-b6fd-e93869169c59 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:08:00.009875 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5309c87c-3ae9-4999-b6fd-e93869169c59 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:08:00.009875 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5309c87c-3ae9-4999-b6fd-e93869169c59 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:08:00.011053 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 711/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 12:08:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:08:00.017036 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4494863-3373-496c-a748-ac558eb768d8 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:08:00.018183 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4494863-3373-496c-a748-ac558eb768d8 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:08:00.018183 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4494863-3373-496c-a748-ac558eb768d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:08:00.103019 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-a4494863-3373-496c-a748-ac558eb768d8 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:08:00.104268 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-a4494863-3373-496c-a748-ac558eb768d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:08:00.211277 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 719/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 12:08:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:08:00.363398 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5309c87c-3ae9-4999-b6fd-e93869169c59 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:08:00.364914 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-5309c87c-3ae9-4999-b6fd-e93869169c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:08:00.414311 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:08:00.533790 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5309c87c-3ae9-4999-b6fd-e93869169c59 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:08:00.651888 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c394081-7852-42cc-8f63-7efe95ae27cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aacd7d6d17e4f10ba5c9107a12beafa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:08:00.652490 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c394081-7852-42cc-8f63-7efe95ae27cd service nova] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:08:00.652770 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c394081-7852-42cc-8f63-7efe95ae27cd service nova] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:08:00.653038 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c394081-7852-42cc-8f63-7efe95ae27cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:08:00.654635 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-4c394081-7852-42cc-8f63-7efe95ae27cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:08:00.661978 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:08:00.744906 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c394081-7852-42cc-8f63-7efe95ae27cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:08:00.746563 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c394081-7852-42cc-8f63-7efe95ae27cd service nova] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:08:00.766045 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 712/1431] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:08:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:08:01.564600 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f8488-2f45-44e3-9c3f-9370c879ef12 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:08:01.565194 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f8488-2f45-44e3-9c3f-9370c879ef12 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:08:01.565194 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f8488-2f45-44e3-9c3f-9370c879ef12 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:08:01.566502 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 720/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 12:08:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:08:01.571424 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba16507-11c0-4672-a60e-dc675afd9b8a None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:08:01.571867 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba16507-11c0-4672-a60e-dc675afd9b8a None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:08:01.571978 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba16507-11c0-4672-a60e-dc675afd9b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:08:01.648638 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-5ba16507-11c0-4672-a60e-dc675afd9b8a None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:08:01.649884 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-5ba16507-11c0-4672-a60e-dc675afd9b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:08:01.715873 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:08:01.745133 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 713/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 12:08:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:08:01.897075 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192f8488-2f45-44e3-9c3f-9370c879ef12 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:08:01.898620 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-192f8488-2f45-44e3-9c3f-9370c879ef12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:08:01.916997 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:08:02.048026 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192f8488-2f45-44e3-9c3f-9370c879ef12 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:08:02.135276 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:08:02.161108 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0e7193-8cca-46a3-bc07-c4050b374176 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aacd7d6d17e4f10ba5c9107a12beafa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:08:02.161707 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e7193-8cca-46a3-bc07-c4050b374176 service nova] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:08:02.161979 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e7193-8cca-46a3-bc07-c4050b374176 service nova] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:08:02.162248 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e7193-8cca-46a3-bc07-c4050b374176 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:08:02.163858 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-6e0e7193-8cca-46a3-bc07-c4050b374176 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:08:02.252062 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0e7193-8cca-46a3-bc07-c4050b374176 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:08:02.254236 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0e7193-8cca-46a3-bc07-c4050b374176 service nova] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:08:02.272334 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 721/1434] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:08:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:08:03.070639 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0419e73-3b23-4ec5-bb0a-0074c97b9726 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:08:03.071168 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0419e73-3b23-4ec5-bb0a-0074c97b9726 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:08:03.071168 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0419e73-3b23-4ec5-bb0a-0074c97b9726 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:08:03.072559 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 714/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 12:08:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:08:03.077398 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5fb20-3333-49e7-b865-71d043d4f355 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:08:03.077790 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5fb20-3333-49e7-b865-71d043d4f355 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:08:03.077947 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5fb20-3333-49e7-b865-71d043d4f355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:08:03.154015 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-1ce5fb20-3333-49e7-b865-71d043d4f355 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:08:03.155214 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-1ce5fb20-3333-49e7-b865-71d043d4f355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:08:03.197423 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:08:03.251065 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 722/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 12:08:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:08:03.399948 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0419e73-3b23-4ec5-bb0a-0074c97b9726 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:08:03.401474 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-a0419e73-3b23-4ec5-bb0a-0074c97b9726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:08:03.543186 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0419e73-3b23-4ec5-bb0a-0074c97b9726 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:08:03.596170 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:08:03.655293 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca63a8a0-a381-489c-932b-418de8a45a00 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aacd7d6d17e4f10ba5c9107a12beafa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:08:03.655833 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63a8a0-a381-489c-932b-418de8a45a00 service nova] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:08:03.656111 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63a8a0-a381-489c-932b-418de8a45a00 service nova] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:08:03.656383 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63a8a0-a381-489c-932b-418de8a45a00 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:08:03.657977 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-ca63a8a0-a381-489c-932b-418de8a45a00 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:08:03.746293 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca63a8a0-a381-489c-932b-418de8a45a00 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:08:03.747939 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca63a8a0-a381-489c-932b-418de8a45a00 service nova] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:08:03.766702 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 715/1437] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:08:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:08:04.567762 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dc803c-b102-416f-acaf-c769dc68f8b8 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:08:04.568334 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dc803c-b102-416f-acaf-c769dc68f8b8 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:08:04.568334 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dc803c-b102-416f-acaf-c769dc68f8b8 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:08:04.569624 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 723/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 12:08:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 12:08:04.574411 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1cbd95-e5fc-4b65-a4a3-33e405f734a7 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:08:04.574796 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1cbd95-e5fc-4b65-a4a3-33e405f734a7 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:08:04.574959 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1cbd95-e5fc-4b65-a4a3-33e405f734a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:08:04.651165 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-ed1cbd95-e5fc-4b65-a4a3-33e405f734a7 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:08:04.652424 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-ed1cbd95-e5fc-4b65-a4a3-33e405f734a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:08:04.654466 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:08:04.754407 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 716/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 12:08:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:08:04.905496 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dc803c-b102-416f-acaf-c769dc68f8b8 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:08:04.907031 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-d9dc803c-b102-416f-acaf-c769dc68f8b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:08:05.055133 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dc803c-b102-416f-acaf-c769dc68f8b8 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:08:05.081414 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:08:05.169782 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a1ce9a-9f36-460c-9227-61c66e4b0eb4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aacd7d6d17e4f10ba5c9107a12beafa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:08:05.170423 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a1ce9a-9f36-460c-9227-61c66e4b0eb4 service nova] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:08:05.170856 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a1ce9a-9f36-460c-9227-61c66e4b0eb4 service nova] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:08:05.171130 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a1ce9a-9f36-460c-9227-61c66e4b0eb4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:08:05.172807 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-63a1ce9a-9f36-460c-9227-61c66e4b0eb4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:08:05.261311 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a1ce9a-9f36-460c-9227-61c66e4b0eb4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:08:05.262929 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a1ce9a-9f36-460c-9227-61c66e4b0eb4 service nova] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:08:05.280547 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 724/1440] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:08:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:08:07.123322 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9839a10b-2cc7-4179-b721-bf2fd6869d26 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:08:07.123874 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9839a10b-2cc7-4179-b721-bf2fd6869d26 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:08:07.123874 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9839a10b-2cc7-4179-b721-bf2fd6869d26 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:08:07.125234 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 717/1441] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 12:08:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:08:07.131087 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93954506-b22a-46a9-a89a-d5a829191f3c None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:08:07.131526 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93954506-b22a-46a9-a89a-d5a829191f3c None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:08:07.131671 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93954506-b22a-46a9-a89a-d5a829191f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:08:07.173113 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:08:07.198019 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-93954506-b22a-46a9-a89a-d5a829191f3c None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:08:07.199244 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-93954506-b22a-46a9-a89a-d5a829191f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:08:07.294604 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 725/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 12:08:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:08:07.579358 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9839a10b-2cc7-4179-b721-bf2fd6869d26 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:08:07.581541 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-9839a10b-2cc7-4179-b721-bf2fd6869d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:08:07.683768 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9839a10b-2cc7-4179-b721-bf2fd6869d26 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:08:07.717121 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:08:07.759824 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e759c857-0cbb-47be-b737-2eed56405bc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:08:07.760425 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e759c857-0cbb-47be-b737-2eed56405bc5 service neutron] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:08:07.760701 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e759c857-0cbb-47be-b737-2eed56405bc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:08:07.760970 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e759c857-0cbb-47be-b737-2eed56405bc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:08:07.762562 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-e759c857-0cbb-47be-b737-2eed56405bc5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:08:07.869182 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e759c857-0cbb-47be-b737-2eed56405bc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:08:07.870632 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e759c857-0cbb-47be-b737-2eed56405bc5 service neutron] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:08:07.888721 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 718/1443] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:08:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:08:09.936593 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138bcf8c-bada-42a4-b237-630cae36dfe7 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:08:09.937147 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138bcf8c-bada-42a4-b237-630cae36dfe7 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:08:09.937147 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138bcf8c-bada-42a4-b237-630cae36dfe7 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:08:09.938512 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 726/1444] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 12:08:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:08:09.944842 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b1378-13dc-44a1-8b46-2e804957fe9d None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:08:09.945303 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b1378-13dc-44a1-8b46-2e804957fe9d None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:08:09.945415 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b1378-13dc-44a1-8b46-2e804957fe9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:08:10.012189 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-058b1378-13dc-44a1-8b46-2e804957fe9d None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:08:10.013287 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-058b1378-13dc-44a1-8b46-2e804957fe9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:08:10.026073 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:08:10.109184 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 719/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 12:08:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:08:10.405979 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138bcf8c-bada-42a4-b237-630cae36dfe7 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:08:10.407547 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-138bcf8c-bada-42a4-b237-630cae36dfe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:08:10.510224 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138bcf8c-bada-42a4-b237-630cae36dfe7 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:08:10.522737 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:08:10.583980 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e11a6f-20a2-47f3-9d6b-a3f23541d35c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:08:10.584574 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e11a6f-20a2-47f3-9d6b-a3f23541d35c service neutron] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:08:10.584848 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e11a6f-20a2-47f3-9d6b-a3f23541d35c service neutron] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:08:10.585113 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e11a6f-20a2-47f3-9d6b-a3f23541d35c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:08:10.586690 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-b0e11a6f-20a2-47f3-9d6b-a3f23541d35c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:08:10.698519 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e11a6f-20a2-47f3-9d6b-a3f23541d35c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:08:10.699894 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e11a6f-20a2-47f3-9d6b-a3f23541d35c service neutron] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:08:10.717605 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 727/1446] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:08:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:08:12.924119 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d6a05c-7236-41e7-8142-33fe82695b1b None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:08:12.924646 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d6a05c-7236-41e7-8142-33fe82695b1b None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:08:12.924848 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d6a05c-7236-41e7-8142-33fe82695b1b None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:08:12.926224 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 720/1447] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 12:08:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:08:12.931460 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7da0d48-5860-47a2-97cd-917f37912a85 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:08:12.931857 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7da0d48-5860-47a2-97cd-917f37912a85 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:08:12.932022 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7da0d48-5860-47a2-97cd-917f37912a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:08:12.979857 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:08:12.998040 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-b7da0d48-5860-47a2-97cd-917f37912a85 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:08:12.999178 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-b7da0d48-5860-47a2-97cd-917f37912a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:08:13.095048 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 728/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 12:08:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:08:13.380290 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d6a05c-7236-41e7-8142-33fe82695b1b None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:08:13.381831 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-08d6a05c-7236-41e7-8142-33fe82695b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:08:13.483676 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d6a05c-7236-41e7-8142-33fe82695b1b None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:08:13.523461 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:08:13.556710 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb9aabe-5b66-46ec-b169-20fa81ec1896 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:08:13.557306 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb9aabe-5b66-46ec-b169-20fa81ec1896 service neutron] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:08:13.557584 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb9aabe-5b66-46ec-b169-20fa81ec1896 service neutron] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:08:13.557854 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb9aabe-5b66-46ec-b169-20fa81ec1896 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:08:13.559473 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-0fb9aabe-5b66-46ec-b169-20fa81ec1896 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:08:13.666677 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb9aabe-5b66-46ec-b169-20fa81ec1896 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:08:13.668057 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb9aabe-5b66-46ec-b169-20fa81ec1896 service neutron] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:08:13.686093 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 721/1449] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:08:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:08:15.716042 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753f63f8-6850-47e9-9a8a-88ac7065ede5 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:08:15.716622 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753f63f8-6850-47e9-9a8a-88ac7065ede5 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:08:15.716622 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753f63f8-6850-47e9-9a8a-88ac7065ede5 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:08:15.718015 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 729/1450] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 12:08:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:08:15.723558 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cec9bd-a8bd-49b2-b41b-80403d2b7df2 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:08:15.723967 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cec9bd-a8bd-49b2-b41b-80403d2b7df2 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:08:15.724104 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cec9bd-a8bd-49b2-b41b-80403d2b7df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:08:15.790255 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-40cec9bd-a8bd-49b2-b41b-80403d2b7df2 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:08:15.791481 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-40cec9bd-a8bd-49b2-b41b-80403d2b7df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:08:15.811566 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:08:15.888870 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 722/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 12:08:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:08:16.171292 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753f63f8-6850-47e9-9a8a-88ac7065ede5 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:08:16.172753 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-753f63f8-6850-47e9-9a8a-88ac7065ede5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:08:16.276766 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753f63f8-6850-47e9-9a8a-88ac7065ede5 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:08:16.296688 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:08:16.350908 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801792ae-a9ac-45d1-b442-3c564f907aec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:08:16.351534 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801792ae-a9ac-45d1-b442-3c564f907aec service neutron] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:08:16.351812 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801792ae-a9ac-45d1-b442-3c564f907aec service neutron] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:08:16.352077 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801792ae-a9ac-45d1-b442-3c564f907aec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:08:16.353655 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-801792ae-a9ac-45d1-b442-3c564f907aec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:08:16.461065 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801792ae-a9ac-45d1-b442-3c564f907aec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:08:16.462411 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801792ae-a9ac-45d1-b442-3c564f907aec service neutron] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:08:16.480482 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 730/1452] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:08:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 12:08:18.446397 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d09b97-de18-4d21-8416-6d9f4820261e None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:08:18.446932 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d09b97-de18-4d21-8416-6d9f4820261e None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:08:18.446932 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d09b97-de18-4d21-8416-6d9f4820261e None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:08:18.448326 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 723/1453] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 12:08:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:08:18.453526 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609c835b-6eb0-4292-9512-421e5da378ad None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:08:18.453920 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609c835b-6eb0-4292-9512-421e5da378ad None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:08:18.454074 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609c835b-6eb0-4292-9512-421e5da378ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:08:18.518909 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:08:18.521670 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-609c835b-6eb0-4292-9512-421e5da378ad None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:08:18.523444 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-609c835b-6eb0-4292-9512-421e5da378ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:08:18.619197 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 731/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 12:08:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:08:18.910867 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d09b97-de18-4d21-8416-6d9f4820261e None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:08:18.912472 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-92d09b97-de18-4d21-8416-6d9f4820261e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:08:19.012738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d09b97-de18-4d21-8416-6d9f4820261e None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:08:19.059613 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:08:19.084120 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88847454-9e61-43da-b214-d4362c6a60eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:08:19.084715 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88847454-9e61-43da-b214-d4362c6a60eb service neutron] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:08:19.084991 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88847454-9e61-43da-b214-d4362c6a60eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:08:19.085256 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88847454-9e61-43da-b214-d4362c6a60eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:08:19.086856 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-88847454-9e61-43da-b214-d4362c6a60eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:08:19.195706 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88847454-9e61-43da-b214-d4362c6a60eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:08:19.197125 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88847454-9e61-43da-b214-d4362c6a60eb service neutron] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:08:19.214641 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 724/1455] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:08:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:08:21.166630 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8941d34f-a693-4e91-a6b0-230768753d43 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:08:21.167175 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8941d34f-a693-4e91-a6b0-230768753d43 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:08:21.167175 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8941d34f-a693-4e91-a6b0-230768753d43 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:08:21.168606 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 732/1456] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 12:08:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:08:21.174926 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dc7cfe-466d-4e03-90e5-664ac51d708f None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:08:21.175380 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dc7cfe-466d-4e03-90e5-664ac51d708f None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:08:21.175505 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dc7cfe-466d-4e03-90e5-664ac51d708f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:08:21.240408 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-18dc7cfe-466d-4e03-90e5-664ac51d708f None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:08:21.241600 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-18dc7cfe-466d-4e03-90e5-664ac51d708f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:08:21.270559 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:08:21.337169 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 725/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 12:08:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:08:21.622215 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8941d34f-a693-4e91-a6b0-230768753d43 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:08:21.623782 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-8941d34f-a693-4e91-a6b0-230768753d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:08:21.726150 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8941d34f-a693-4e91-a6b0-230768753d43 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:08:21.758133 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:08:21.803984 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6bc492-56ab-49a0-9af1-9953053c62c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:08:21.804578 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6bc492-56ab-49a0-9af1-9953053c62c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:08:21.804852 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6bc492-56ab-49a0-9af1-9953053c62c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:08:21.805120 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6bc492-56ab-49a0-9af1-9953053c62c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:08:21.806696 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-eb6bc492-56ab-49a0-9af1-9953053c62c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:08:21.913783 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6bc492-56ab-49a0-9af1-9953053c62c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:08:21.915235 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6bc492-56ab-49a0-9af1-9953053c62c0 service neutron] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:08:21.933207 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 733/1458] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:08:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:08:23.887146 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3157d922-76f2-432b-b956-6f9629257bfd None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:08:23.887703 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3157d922-76f2-432b-b956-6f9629257bfd None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:08:23.887825 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3157d922-76f2-432b-b956-6f9629257bfd None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:08:23.889140 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 726/1459] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 12:08:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:08:23.894943 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f399e139-a575-4490-afc0-5d69cde9d00d None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:08:23.895373 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f399e139-a575-4490-afc0-5d69cde9d00d None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:08:23.895557 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f399e139-a575-4490-afc0-5d69cde9d00d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:08:23.962508 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-f399e139-a575-4490-afc0-5d69cde9d00d None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:08:23.963778 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-f399e139-a575-4490-afc0-5d69cde9d00d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:08:23.972855 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:08:24.062914 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 734/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 12:08:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:08:24.346241 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3157d922-76f2-432b-b956-6f9629257bfd None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:08:24.347875 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-3157d922-76f2-432b-b956-6f9629257bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:08:24.451346 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3157d922-76f2-432b-b956-6f9629257bfd None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:08:24.507028 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:08:24.526059 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105d72a2-121c-44cf-9f03-212ea6ff332d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:08:24.526659 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d72a2-121c-44cf-9f03-212ea6ff332d service neutron] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:08:24.526939 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d72a2-121c-44cf-9f03-212ea6ff332d service neutron] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:08:24.527239 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d72a2-121c-44cf-9f03-212ea6ff332d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:08:24.528856 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-105d72a2-121c-44cf-9f03-212ea6ff332d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:08:24.638827 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105d72a2-121c-44cf-9f03-212ea6ff332d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:08:24.640292 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105d72a2-121c-44cf-9f03-212ea6ff332d service neutron] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:08:24.668661 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 727/1461] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:08:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:08:26.612037 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4207378-993e-4cfb-98a0-9071739925ab None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:08:26.612614 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4207378-993e-4cfb-98a0-9071739925ab None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:08:26.612614 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4207378-993e-4cfb-98a0-9071739925ab None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:08:26.613922 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 735/1462] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 12:08:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:08:26.619081 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6265d563-de6d-4d34-8648-6c8f4df43fc1 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:08:26.619511 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6265d563-de6d-4d34-8648-6c8f4df43fc1 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:08:26.619678 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6265d563-de6d-4d34-8648-6c8f4df43fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:08:26.684552 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-6265d563-de6d-4d34-8648-6c8f4df43fc1 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:08:26.685735 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-6265d563-de6d-4d34-8648-6c8f4df43fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:08:26.723610 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:08:26.783555 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 728/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 12:08:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:08:27.069575 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4207378-993e-4cfb-98a0-9071739925ab None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:08:27.071117 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-d4207378-993e-4cfb-98a0-9071739925ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:08:27.172740 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4207378-993e-4cfb-98a0-9071739925ab None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:08:27.209301 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:08:27.244392 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b6cd3a-0420-4a11-86a0-8a87d6f8578e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:08:27.244984 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b6cd3a-0420-4a11-86a0-8a87d6f8578e service neutron] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:08:27.245257 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b6cd3a-0420-4a11-86a0-8a87d6f8578e service neutron] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:08:27.245523 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b6cd3a-0420-4a11-86a0-8a87d6f8578e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:08:27.247089 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-50b6cd3a-0420-4a11-86a0-8a87d6f8578e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:08:27.354397 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b6cd3a-0420-4a11-86a0-8a87d6f8578e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:08:27.355788 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b6cd3a-0420-4a11-86a0-8a87d6f8578e service neutron] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:08:27.373242 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 736/1464] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:08:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:08:29.285769 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffecad0-6bb7-4c56-8ba4-7ba980a25097 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:08:29.286304 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffecad0-6bb7-4c56-8ba4-7ba980a25097 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:08:29.286304 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffecad0-6bb7-4c56-8ba4-7ba980a25097 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:08:29.287680 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 729/1465] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 12:08:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:08:29.292874 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d935a3-4790-41da-b79b-e59180e525e4 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:08:29.293253 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d935a3-4790-41da-b79b-e59180e525e4 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:08:29.293438 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d935a3-4790-41da-b79b-e59180e525e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:08:29.359862 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-84d935a3-4790-41da-b79b-e59180e525e4 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:08:29.361579 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-84d935a3-4790-41da-b79b-e59180e525e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:08:29.382059 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:08:29.459571 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 737/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 12:08:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:08:29.743463 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffecad0-6bb7-4c56-8ba4-7ba980a25097 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:08:29.745280 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-1ffecad0-6bb7-4c56-8ba4-7ba980a25097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:08:29.848613 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffecad0-6bb7-4c56-8ba4-7ba980a25097 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:08:29.914103 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:08:29.921469 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b15a50-7960-4a30-85a8-5ba2ab3c1ebd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:08:29.922071 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b15a50-7960-4a30-85a8-5ba2ab3c1ebd service neutron] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:08:29.922350 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b15a50-7960-4a30-85a8-5ba2ab3c1ebd service neutron] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:08:29.922624 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b15a50-7960-4a30-85a8-5ba2ab3c1ebd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:08:29.924273 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-f5b15a50-7960-4a30-85a8-5ba2ab3c1ebd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:08:30.040307 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b15a50-7960-4a30-85a8-5ba2ab3c1ebd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:08:30.041645 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b15a50-7960-4a30-85a8-5ba2ab3c1ebd service neutron] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:08:30.059447 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 730/1467] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:08:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:08:31.971390 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d25eea-7975-4d46-a649-b203933f4e0c None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:08:31.971985 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d25eea-7975-4d46-a649-b203933f4e0c None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:08:31.971985 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d25eea-7975-4d46-a649-b203933f4e0c None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:08:31.973279 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 738/1468] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 12:08:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:08:31.978950 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e39d2d1-5748-4491-aa4c-15fe2bf3cc91 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:08:31.979450 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e39d2d1-5748-4491-aa4c-15fe2bf3cc91 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:08:31.979610 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e39d2d1-5748-4491-aa4c-15fe2bf3cc91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:08:32.051345 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-5e39d2d1-5748-4491-aa4c-15fe2bf3cc91 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:08:32.052486 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-5e39d2d1-5748-4491-aa4c-15fe2bf3cc91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:08:32.099971 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:08:32.148370 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 731/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 12:08:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:08:32.431394 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d25eea-7975-4d46-a649-b203933f4e0c None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:08:32.432920 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-26d25eea-7975-4d46-a649-b203933f4e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:08:32.535057 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d25eea-7975-4d46-a649-b203933f4e0c None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:08:32.582708 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:08:32.608724 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558968e6-5cfd-40e2-89f8-045417ca1670 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:08:32.609328 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558968e6-5cfd-40e2-89f8-045417ca1670 service neutron] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:08:32.609601 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558968e6-5cfd-40e2-89f8-045417ca1670 service neutron] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:08:32.609869 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558968e6-5cfd-40e2-89f8-045417ca1670 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:08:32.612173 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-558968e6-5cfd-40e2-89f8-045417ca1670 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:08:32.720019 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558968e6-5cfd-40e2-89f8-045417ca1670 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:08:32.721371 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558968e6-5cfd-40e2-89f8-045417ca1670 service neutron] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:08:32.740086 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 739/1470] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:08:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:08:42.146560 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=22 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:08:42.737780 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=66 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:08:52.298045 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6309123b-aab0-4db7-92d2-ea580723eb8d None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:08:52.298611 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6309123b-aab0-4db7-92d2-ea580723eb8d None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:08:52.298611 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6309123b-aab0-4db7-92d2-ea580723eb8d None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:08:52.299975 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 732/1471] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:08:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:08:52.305173 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82598649-cacf-437c-a685-c1a0577c3f48 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:08:52.305550 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82598649-cacf-437c-a685-c1a0577c3f48 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:08:52.305730 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82598649-cacf-437c-a685-c1a0577c3f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:08:52.371751 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-82598649-cacf-437c-a685-c1a0577c3f48 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:08:52.372966 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-82598649-cacf-437c-a685-c1a0577c3f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:08:52.468156 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 740/1472] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:08:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:08:52.769207 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb0729d-5e15-411e-b257-f33f36b22c81 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:08:52.769496 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb0729d-5e15-411e-b257-f33f36b22c81 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:08:52.769776 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb0729d-5e15-411e-b257-f33f36b22c81 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:08:52.771099 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 733/1473] 10.222.0.61 () {54 vars in 967 bytes} [Fri Aug 26 12:08:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:08:52.776312 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3d434-03d8-41c3-ac47-dba9450c8f63 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:08:52.776312 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3d434-03d8-41c3-ac47-dba9450c8f63 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:08:52.776497 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3d434-03d8-41c3-ac47-dba9450c8f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:08:52.851795 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-baf3d434-03d8-41c3-ac47-dba9450c8f63 None None] MFA Rules not processed for user `152fd7c708114f249b9c960165f0fdfe`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:08:52.852991 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-baf3d434-03d8-41c3-ac47-dba9450c8f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:08:52.872904 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:08:52.929040 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 741/1474] 10.222.0.61 () {58 vars in 1064 bytes} [Fri Aug 26 12:08:52 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:08:52.936478 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb0729d-5e15-411e-b257-f33f36b22c81 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:08:52.938169 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-2bb0729d-5e15-411e-b257-f33f36b22c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:08:53.040073 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb0729d-5e15-411e-b257-f33f36b22c81 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:08:53.110634 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a51c83-2cd1-46d5-bc88-47047d8818dc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152fd7c708114f249b9c960165f0fdfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:08:53.111244 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a51c83-2cd1-46d5-bc88-47047d8818dc service glance] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:08:53.111525 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a51c83-2cd1-46d5-bc88-47047d8818dc service glance] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:08:53.111789 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a51c83-2cd1-46d5-bc88-47047d8818dc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:08:53.113385 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-37a51c83-2cd1-46d5-bc88-47047d8818dc service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:08:53.166235 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:08:53.222159 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a51c83-2cd1-46d5-bc88-47047d8818dc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:08:53.223635 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a51c83-2cd1-46d5-bc88-47047d8818dc service glance] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:08:53.241895 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 734/1475] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:08:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:08:55.216899 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e48c7-a49a-42d0-b0b2-b7b0cfa1d0f1 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:08:55.217480 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e48c7-a49a-42d0-b0b2-b7b0cfa1d0f1 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:08:55.217480 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e48c7-a49a-42d0-b0b2-b7b0cfa1d0f1 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:08:55.219596 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 742/1476] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:08:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:08:55.224691 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eaaedf-09c8-4a6c-acc6-b16c2eca9f6a None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:08:55.225127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eaaedf-09c8-4a6c-acc6-b16c2eca9f6a None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:08:55.225259 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eaaedf-09c8-4a6c-acc6-b16c2eca9f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:08:55.289673 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-47eaaedf-09c8-4a6c-acc6-b16c2eca9f6a None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:08:55.290849 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-47eaaedf-09c8-4a6c-acc6-b16c2eca9f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:08:55.384609 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 735/1477] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:08:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:08:55.670063 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088e48c7-a49a-42d0-b0b2-b7b0cfa1d0f1 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:08:55.672184 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-088e48c7-a49a-42d0-b0b2-b7b0cfa1d0f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:08:55.915510 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088e48c7-a49a-42d0-b0b2-b7b0cfa1d0f1 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:08:55.963571 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:08:56.008027 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eed1209-10b8-45b7-a44a-fb4860ad0844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:08:56.008627 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eed1209-10b8-45b7-a44a-fb4860ad0844 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:08:56.008906 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eed1209-10b8-45b7-a44a-fb4860ad0844 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:08:56.009177 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eed1209-10b8-45b7-a44a-fb4860ad0844 admin admin] PATH_INFO: `/v3/projects` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:08:56.009810 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eed1209-10b8-45b7-a44a-fb4860ad0844 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:08:56.011559 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: WARNING py.warnings [None req-2eed1209-10b8-45b7-a44a-fb4860ad0844 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 12:08:56.011559 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: warnings.warn(msg) Aug 26 12:08:56.011559 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: Aug 26 12:08:56.012689 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eed1209-10b8-45b7-a44a-fb4860ad0844 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:08:56.031841 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 743/1478] 10.222.0.61 () {56 vars in 1108 bytes} [Fri Aug 26 12:08:55 2022] GET /identity/v3/projects => generated 2553 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:08:57.982194 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df82a47-cfb6-4703-8902-185ac66544b0 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:08:57.982749 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df82a47-cfb6-4703-8902-185ac66544b0 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:08:57.982749 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df82a47-cfb6-4703-8902-185ac66544b0 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:08:57.984232 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 736/1479] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:08:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 12:08:57.989742 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfde475-11e7-42a2-8c7d-3161db386f73 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:08:57.990166 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfde475-11e7-42a2-8c7d-3161db386f73 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:08:57.990280 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfde475-11e7-42a2-8c7d-3161db386f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:08:58.055875 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-5bfde475-11e7-42a2-8c7d-3161db386f73 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:08:58.056996 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-5bfde475-11e7-42a2-8c7d-3161db386f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:08:58.158334 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 744/1480] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:08:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:08:58.455383 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df82a47-cfb6-4703-8902-185ac66544b0 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:08:58.456936 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-7df82a47-cfb6-4703-8902-185ac66544b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:08:58.466771 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:08:58.601622 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df82a47-cfb6-4703-8902-185ac66544b0 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:08:58.681015 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:08:58.716330 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5809dff0-65f6-4d31-8550-13c0e25dafa3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aacd7d6d17e4f10ba5c9107a12beafa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:08:58.716845 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5809dff0-65f6-4d31-8550-13c0e25dafa3 service nova] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:08:58.717122 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5809dff0-65f6-4d31-8550-13c0e25dafa3 service nova] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:08:58.717393 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5809dff0-65f6-4d31-8550-13c0e25dafa3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:08:58.719118 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-5809dff0-65f6-4d31-8550-13c0e25dafa3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:08:58.809537 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5809dff0-65f6-4d31-8550-13c0e25dafa3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:08:58.811122 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5809dff0-65f6-4d31-8550-13c0e25dafa3 service nova] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:08:58.829201 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 737/1481] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:08:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:09:00.912039 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306e4b5-ac7e-49a3-9932-411f3049aa94 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:09:00.912552 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306e4b5-ac7e-49a3-9932-411f3049aa94 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:09:00.912552 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306e4b5-ac7e-49a3-9932-411f3049aa94 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:09:00.913843 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 745/1482] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:09:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:09:00.919020 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f613fa-3c8c-4218-b67d-5411675e9d96 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:09:00.919444 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f613fa-3c8c-4218-b67d-5411675e9d96 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:09:00.919607 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f613fa-3c8c-4218-b67d-5411675e9d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:09:00.986107 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-d3f613fa-3c8c-4218-b67d-5411675e9d96 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:09:00.987370 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-d3f613fa-3c8c-4218-b67d-5411675e9d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:09:01.033586 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:09:01.083889 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 738/1483] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:09:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:09:01.371417 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2306e4b5-ac7e-49a3-9932-411f3049aa94 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:09:01.372936 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-2306e4b5-ac7e-49a3-9932-411f3049aa94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:09:01.397322 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:09:01.479243 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2306e4b5-ac7e-49a3-9932-411f3049aa94 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:09:01.551097 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed8b206-4366-47fc-aed2-e1b833ccdc9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:09:01.551711 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed8b206-4366-47fc-aed2-e1b833ccdc9c service neutron] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:09:01.551992 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed8b206-4366-47fc-aed2-e1b833ccdc9c service neutron] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:09:01.552259 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed8b206-4366-47fc-aed2-e1b833ccdc9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:09:01.553870 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-8ed8b206-4366-47fc-aed2-e1b833ccdc9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:09:01.635227 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:09:01.670049 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed8b206-4366-47fc-aed2-e1b833ccdc9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:09:01.672253 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed8b206-4366-47fc-aed2-e1b833ccdc9c service neutron] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:09:01.690777 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 746/1484] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:09:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:09:03.789221 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f3091-ce8b-4f95-ab22-aea9cf8ceacf None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:09:03.789740 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f3091-ce8b-4f95-ab22-aea9cf8ceacf None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:09:03.789846 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f3091-ce8b-4f95-ab22-aea9cf8ceacf None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:09:03.791278 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 739/1485] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:09:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:09:03.796673 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e53b8f-8c38-4be9-b08d-0df66bcb5e2d None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:09:03.797114 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e53b8f-8c38-4be9-b08d-0df66bcb5e2d None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:09:03.797215 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e53b8f-8c38-4be9-b08d-0df66bcb5e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:09:03.862828 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-71e53b8f-8c38-4be9-b08d-0df66bcb5e2d None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:09:03.864047 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-71e53b8f-8c38-4be9-b08d-0df66bcb5e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:09:03.960195 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 747/1486] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:09:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:09:04.250682 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4f3091-ce8b-4f95-ab22-aea9cf8ceacf None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:09:04.252313 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-9e4f3091-ce8b-4f95-ab22-aea9cf8ceacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:09:04.352967 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4f3091-ce8b-4f95-ab22-aea9cf8ceacf None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:09:04.422785 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:09:04.424443 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7764a9a-0822-46df-9e33-75a83fa3422f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:09:04.425131 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7764a9a-0822-46df-9e33-75a83fa3422f service neutron] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:09:04.425426 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7764a9a-0822-46df-9e33-75a83fa3422f service neutron] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:09:04.425673 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7764a9a-0822-46df-9e33-75a83fa3422f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:09:04.427314 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-e7764a9a-0822-46df-9e33-75a83fa3422f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:09:04.535141 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7764a9a-0822-46df-9e33-75a83fa3422f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:09:04.536535 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7764a9a-0822-46df-9e33-75a83fa3422f service neutron] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:09:04.555260 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 740/1487] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:09:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:09:06.599265 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ff301-c0b5-4dbd-849f-45af88292eaf None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:09:06.599827 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ff301-c0b5-4dbd-849f-45af88292eaf None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:09:06.599827 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ff301-c0b5-4dbd-849f-45af88292eaf None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:09:06.601162 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 748/1488] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:09:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:09:06.606299 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089248a-3c22-4fc6-80f1-5554e83629b0 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:09:06.606715 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089248a-3c22-4fc6-80f1-5554e83629b0 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:09:06.606846 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089248a-3c22-4fc6-80f1-5554e83629b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:09:06.672609 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-9089248a-3c22-4fc6-80f1-5554e83629b0 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:09:06.673794 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-9089248a-3c22-4fc6-80f1-5554e83629b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:09:06.725331 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:09:06.770071 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 741/1489] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:09:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:09:07.059477 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81ff301-c0b5-4dbd-849f-45af88292eaf None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:09:07.061015 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-f81ff301-c0b5-4dbd-849f-45af88292eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:09:07.097195 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:09:07.164868 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81ff301-c0b5-4dbd-849f-45af88292eaf None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:09:07.241234 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a86f5-ace9-4d6a-89aa-1eb45ac9d04b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:09:07.241821 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a86f5-ace9-4d6a-89aa-1eb45ac9d04b service neutron] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:09:07.242093 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a86f5-ace9-4d6a-89aa-1eb45ac9d04b service neutron] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:09:07.242356 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a86f5-ace9-4d6a-89aa-1eb45ac9d04b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:09:07.243965 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-b59a86f5-ace9-4d6a-89aa-1eb45ac9d04b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:09:07.322615 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:09:07.352481 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59a86f5-ace9-4d6a-89aa-1eb45ac9d04b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:09:07.353907 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59a86f5-ace9-4d6a-89aa-1eb45ac9d04b service neutron] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:09:07.371587 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 749/1490] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:09:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:09:09.470293 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30412021-5c46-4942-8d13-cb464713872d None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:09:09.470834 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30412021-5c46-4942-8d13-cb464713872d None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:09:09.470834 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30412021-5c46-4942-8d13-cb464713872d None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:09:09.472213 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 742/1491] 10.222.0.61 () {54 vars in 927 bytes} [Fri Aug 26 12:09:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:09:09.477556 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be9d52c-0de3-43b0-ad05-c96785d65fcd None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:09:09.477950 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be9d52c-0de3-43b0-ad05-c96785d65fcd None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:09:09.478101 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be9d52c-0de3-43b0-ad05-c96785d65fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:09:09.543574 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-6be9d52c-0de3-43b0-ad05-c96785d65fcd None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:09:09.544755 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-6be9d52c-0de3-43b0-ad05-c96785d65fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:09:09.640075 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 750/1492] 10.222.0.61 () {58 vars in 1024 bytes} [Fri Aug 26 12:09:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:09:09.934638 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30412021-5c46-4942-8d13-cb464713872d None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:09:09.936787 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-30412021-5c46-4942-8d13-cb464713872d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:09:10.038573 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30412021-5c46-4942-8d13-cb464713872d None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:09:10.109763 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b600ad09-eadb-4b99-bd5f-a23b1bc055ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:09:10.110354 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600ad09-eadb-4b99-bd5f-a23b1bc055ba service neutron] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:09:10.110630 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600ad09-eadb-4b99-bd5f-a23b1bc055ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:09:10.110898 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600ad09-eadb-4b99-bd5f-a23b1bc055ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:09:10.112508 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-b600ad09-eadb-4b99-bd5f-a23b1bc055ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:09:10.121227 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:09:10.220278 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b600ad09-eadb-4b99-bd5f-a23b1bc055ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:09:10.221702 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b600ad09-eadb-4b99-bd5f-a23b1bc055ba service neutron] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:09:10.240407 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 743/1493] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:09:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:09:19.637817 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=82 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:09:20.238134 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=50 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:10:32.538182 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e69876-d007-41fc-b633-2e590c44df39 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:10:32.538766 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e69876-d007-41fc-b633-2e590c44df39 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:10:32.538766 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e69876-d007-41fc-b633-2e590c44df39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:10:32.621223 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-d3e69876-d007-41fc-b633-2e590c44df39 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:10:32.622693 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-d3e69876-d007-41fc-b633-2e590c44df39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:10:32.719264 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 751/1494] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:10:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:10:32.727311 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc16268b-f163-44d5-bb35-cc50c7a328b1 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:10:32.727793 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc16268b-f163-44d5-bb35-cc50c7a328b1 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:10:32.727884 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc16268b-f163-44d5-bb35-cc50c7a328b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:10:32.805584 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-cc16268b-f163-44d5-bb35-cc50c7a328b1 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:10:32.806707 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-cc16268b-f163-44d5-bb35-cc50c7a328b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:10:32.901912 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 744/1495] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:10:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:10:32.906619 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e69876-d007-41fc-b633-2e590c44df39 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:10:32.907831 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-d3e69876-d007-41fc-b633-2e590c44df39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:10:32.918991 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e69876-d007-41fc-b633-2e590c44df39 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:10:32.978823 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:10:33.012250 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2209f894-d0b6-47ca-9b92-db00517df249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:10:33.013419 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2209f894-d0b6-47ca-9b92-db00517df249 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:10:33.013696 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2209f894-d0b6-47ca-9b92-db00517df249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:10:33.013965 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2209f894-d0b6-47ca-9b92-db00517df249 admin admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:10:33.014624 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2209f894-d0b6-47ca-9b92-db00517df249 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:10:33.014920 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2209f894-d0b6-47ca-9b92-db00517df249 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:10:33.015953 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: WARNING py.warnings [None req-2209f894-d0b6-47ca-9b92-db00517df249 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 26 12:10:33.015953 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: warnings.warn(msg) Aug 26 12:10:33.015953 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: Aug 26 12:10:33.016832 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2209f894-d0b6-47ca-9b92-db00517df249 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:10:33.027814 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 752/1496] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:10:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:10:33.036182 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc16268b-f163-44d5-bb35-cc50c7a328b1 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:10:33.045614 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc16268b-f163-44d5-bb35-cc50c7a328b1 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:10:33.105231 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:10:33.142556 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98741787-3b8e-4cf4-84e8-0d43ceba9b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:10:33.143162 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98741787-3b8e-4cf4-84e8-0d43ceba9b89 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:10:33.143479 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98741787-3b8e-4cf4-84e8-0d43ceba9b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:10:33.143748 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98741787-3b8e-4cf4-84e8-0d43ceba9b89 admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:10:33.144373 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98741787-3b8e-4cf4-84e8-0d43ceba9b89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:10:33.146132 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: WARNING py.warnings [None req-98741787-3b8e-4cf4-84e8-0d43ceba9b89 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 12:10:33.146132 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: warnings.warn(msg) Aug 26 12:10:33.146132 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: Aug 26 12:10:33.147370 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98741787-3b8e-4cf4-84e8-0d43ceba9b89 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:10:33.203753 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 745/1497] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:10:33 2022] POST /identity/v3/projects => generated 379 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:10:33.208144 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2209f894-d0b6-47ca-9b92-db00517df249 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:10:33.209767 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-2209f894-d0b6-47ca-9b92-db00517df249 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:10:33.242573 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2209f894-d0b6-47ca-9b92-db00517df249 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:10:33.333391 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b550279-50b4-4c3c-bb92-2b3170f8e0be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:10:33.334000 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b550279-50b4-4c3c-bb92-2b3170f8e0be admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:10:33.334274 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b550279-50b4-4c3c-bb92-2b3170f8e0be admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:10:33.334538 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b550279-50b4-4c3c-bb92-2b3170f8e0be admin admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:10:33.335143 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b550279-50b4-4c3c-bb92-2b3170f8e0be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:10:33.336152 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: WARNING py.warnings [None req-7b550279-50b4-4c3c-bb92-2b3170f8e0be admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 12:10:33.336152 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: warnings.warn(msg) Aug 26 12:10:33.336152 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: Aug 26 12:10:33.337343 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b550279-50b4-4c3c-bb92-2b3170f8e0be admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:10:33.379475 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 753/1498] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:10:33 2022] POST /identity/v3/users => generated 290 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:10:33.383481 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98741787-3b8e-4cf4-84e8-0d43ceba9b89 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:10:33.385017 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-98741787-3b8e-4cf4-84e8-0d43ceba9b89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:10:33.446072 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98741787-3b8e-4cf4-84e8-0d43ceba9b89 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:10:33.527007 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:10:33.538897 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e222c17-d7b6-4566-9bf8-c6a1fbdc79e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:10:33.539522 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e222c17-d7b6-4566-9bf8-c6a1fbdc79e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:10:33.539805 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e222c17-d7b6-4566-9bf8-c6a1fbdc79e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:10:33.540079 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e222c17-d7b6-4566-9bf8-c6a1fbdc79e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:10:33.540721 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e222c17-d7b6-4566-9bf8-c6a1fbdc79e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:10:33.541727 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: WARNING py.warnings [None req-2e222c17-d7b6-4566-9bf8-c6a1fbdc79e3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 26 12:10:33.541727 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: warnings.warn(msg) Aug 26 12:10:33.541727 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: Aug 26 12:10:33.542623 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e222c17-d7b6-4566-9bf8-c6a1fbdc79e3 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:10:33.555253 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 746/1499] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:10:33 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:10:33.559364 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b550279-50b4-4c3c-bb92-2b3170f8e0be admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:10:33.561136 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-7b550279-50b4-4c3c-bb92-2b3170f8e0be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:10:33.574684 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b550279-50b4-4c3c-bb92-2b3170f8e0be admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:10:33.600249 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:10:33.672494 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140d068b-6918-4c3a-9a90-046e028cf560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:10:33.673017 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d068b-6918-4c3a-9a90-046e028cf560 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:10:33.673294 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d068b-6918-4c3a-9a90-046e028cf560 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:10:33.673558 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d068b-6918-4c3a-9a90-046e028cf560 admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:10:33.674189 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140d068b-6918-4c3a-9a90-046e028cf560 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:10:33.675183 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: WARNING py.warnings [None req-140d068b-6918-4c3a-9a90-046e028cf560 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 26 12:10:33.675183 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: warnings.warn(msg) Aug 26 12:10:33.675183 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: Aug 26 12:10:33.676094 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140d068b-6918-4c3a-9a90-046e028cf560 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:10:33.688441 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 754/1500] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:10:33 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:10:33.692756 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e222c17-d7b6-4566-9bf8-c6a1fbdc79e3 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:10:33.694277 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-2e222c17-d7b6-4566-9bf8-c6a1fbdc79e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:10:33.726981 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e222c17-d7b6-4566-9bf8-c6a1fbdc79e3 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:10:33.823013 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a56e823-a7d6-47c2-9d7d-690429fb02bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:10:33.823691 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a56e823-a7d6-47c2-9d7d-690429fb02bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:10:33.823968 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a56e823-a7d6-47c2-9d7d-690429fb02bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:10:33.824238 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a56e823-a7d6-47c2-9d7d-690429fb02bc admin admin] PATH_INFO: `/v3/projects/a5ab4a0a80954a34a11b0db3ea81b14f/users/290340256678451787f68fbf28208ca1/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:10:33.846624 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a56e823-a7d6-47c2-9d7d-690429fb02bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5ab4a0a80954a34a11b0db3ea81b14f, user_id=290340256678451787f68fbf28208ca1, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:10:33.847695 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: WARNING py.warnings [None req-1a56e823-a7d6-47c2-9d7d-690429fb02bc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 12:10:33.847695 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: warnings.warn(msg) Aug 26 12:10:33.847695 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: Aug 26 12:10:33.849636 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a56e823-a7d6-47c2-9d7d-690429fb02bc admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:10:33.992707 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 747/1501] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:10:33 2022] PUT /identity/v3/projects/a5ab4a0a80954a34a11b0db3ea81b14f/users/290340256678451787f68fbf28208ca1/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:10:34.005533 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9558b170-96f2-456b-b20a-c5e718c7a0ee None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:10:34.005820 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9558b170-96f2-456b-b20a-c5e718c7a0ee None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:10:34.006100 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9558b170-96f2-456b-b20a-c5e718c7a0ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:10:34.063903 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-9558b170-96f2-456b-b20a-c5e718c7a0ee None None] MFA Rules not processed for user `290340256678451787f68fbf28208ca1`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:10:34.065112 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-9558b170-96f2-456b-b20a-c5e718c7a0ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:10:34.144800 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 755/1502] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:10:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:10:34.158443 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd54b36-832e-4fa3-825a-64c29ec0df7e None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:10:34.158729 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd54b36-832e-4fa3-825a-64c29ec0df7e None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:10:34.158998 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd54b36-832e-4fa3-825a-64c29ec0df7e None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:10:34.160353 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 748/1503] 10.222.0.61 () {54 vars in 964 bytes} [Fri Aug 26 12:10:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:10:34.164998 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582f12a6-3580-4305-b49d-f35b76037016 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:10:34.165284 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582f12a6-3580-4305-b49d-f35b76037016 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:10:34.165563 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582f12a6-3580-4305-b49d-f35b76037016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:10:34.196982 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:10:34.241428 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-582f12a6-3580-4305-b49d-f35b76037016 None None] MFA Rules not processed for user `51239ac314e245e8a5706e65b133db3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:10:34.242618 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-582f12a6-3580-4305-b49d-f35b76037016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:10:34.320057 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 756/1504] 10.222.0.61 () {58 vars in 1061 bytes} [Fri Aug 26 12:10:34 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:10:34.328357 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd54b36-832e-4fa3-825a-64c29ec0df7e None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:10:34.329848 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-3cd54b36-832e-4fa3-825a-64c29ec0df7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:10:34.378471 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:10:34.431724 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd54b36-832e-4fa3-825a-64c29ec0df7e None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:10:34.504005 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a80f45-47cd-4c01-a756-ee5c0a714523 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:10:34.504596 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a80f45-47cd-4c01-a756-ee5c0a714523 service cinder] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:10:34.504870 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a80f45-47cd-4c01-a756-ee5c0a714523 service cinder] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:10:34.505138 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a80f45-47cd-4c01-a756-ee5c0a714523 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:10:34.506712 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-32a80f45-47cd-4c01-a756-ee5c0a714523 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:10:34.601758 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a80f45-47cd-4c01-a756-ee5c0a714523 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:10:34.603096 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a80f45-47cd-4c01-a756-ee5c0a714523 service cinder] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:10:34.607167 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:10:34.623848 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 749/1505] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:10:34 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:10:34.667375 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721bee33-087d-48dd-82ec-26b7da3ca35f None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:10:34.667773 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721bee33-087d-48dd-82ec-26b7da3ca35f None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:10:34.667921 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721bee33-087d-48dd-82ec-26b7da3ca35f None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:10:34.669882 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 757/1506] 10.222.0.61 () {52 vars in 830 bytes} [Fri Aug 26 12:10:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:10:34.673502 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a80f45-47cd-4c01-a756-ee5c0a714523 service cinder] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:10:34.674710 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-32a80f45-47cd-4c01-a756-ee5c0a714523 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:10:34.778341 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a80f45-47cd-4c01-a756-ee5c0a714523 service cinder] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:10:34.871077 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55de5336-a552-4222-aced-9d1afd775efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:10:34.872552 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de5336-a552-4222-aced-9d1afd775efc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:10:34.872791 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de5336-a552-4222-aced-9d1afd775efc admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:10:34.873059 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de5336-a552-4222-aced-9d1afd775efc admin admin] PATH_INFO: `/v3/users/290340256678451787f68fbf28208ca1` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:10:34.886718 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55de5336-a552-4222-aced-9d1afd775efc admin admin] RBAC: Authorizing `identity:delete_user(user_id=290340256678451787f68fbf28208ca1)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:10:34.887849 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: WARNING py.warnings [None req-55de5336-a552-4222-aced-9d1afd775efc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 12:10:34.887849 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: warnings.warn(msg) Aug 26 12:10:34.887849 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: Aug 26 12:10:34.889051 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55de5336-a552-4222-aced-9d1afd775efc admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:10:34.898614 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:10:34.958134 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-55de5336-a552-4222-aced-9d1afd775efc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '290340256678451787f68fbf28208ca1'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:10:34.967675 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-55de5336-a552-4222-aced-9d1afd775efc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '290340256678451787f68fbf28208ca1'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:10:34.982377 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-55de5336-a552-4222-aced-9d1afd775efc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '290340256678451787f68fbf28208ca1'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:10:34.982911 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-55de5336-a552-4222-aced-9d1afd775efc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '290340256678451787f68fbf28208ca1'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:10:35.005541 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-55de5336-a552-4222-aced-9d1afd775efc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '290340256678451787f68fbf28208ca1'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:10:35.011240 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 750/1507] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:10:34 2022] DELETE /identity/v3/users/290340256678451787f68fbf28208ca1 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:10:35.015915 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52464294-ae0a-4abe-b603-f7d638e15d4c None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:10:35.016204 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52464294-ae0a-4abe-b603-f7d638e15d4c None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:10:35.016480 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52464294-ae0a-4abe-b603-f7d638e15d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:10:35.073619 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-52464294-ae0a-4abe-b603-f7d638e15d4c None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:10:35.074865 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-52464294-ae0a-4abe-b603-f7d638e15d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:10:35.150617 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:10:35.164002 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 758/1508] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:10:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:10:35.174329 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55de5336-a552-4222-aced-9d1afd775efc admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:10:35.175601 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-55de5336-a552-4222-aced-9d1afd775efc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:10:35.272495 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55de5336-a552-4222-aced-9d1afd775efc admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:10:35.326592 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:10:35.348258 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac86d0e1-1c19-49e6-9a59-9cc9ce055c27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:10:35.348871 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac86d0e1-1c19-49e6-9a59-9cc9ce055c27 service neutron] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:10:35.349151 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac86d0e1-1c19-49e6-9a59-9cc9ce055c27 service neutron] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:10:35.349426 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac86d0e1-1c19-49e6-9a59-9cc9ce055c27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:10:35.351042 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-ac86d0e1-1c19-49e6-9a59-9cc9ce055c27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:10:35.459453 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac86d0e1-1c19-49e6-9a59-9cc9ce055c27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:10:35.460826 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac86d0e1-1c19-49e6-9a59-9cc9ce055c27 service neutron] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:10:35.479525 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 751/1509] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:10:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:10:35.831333 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52464294-ae0a-4abe-b603-f7d638e15d4c None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:10:35.832579 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-52464294-ae0a-4abe-b603-f7d638e15d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:10:35.844627 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52464294-ae0a-4abe-b603-f7d638e15d4c None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:10:35.942820 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c668ddcf-4df5-4f8a-9e6d-86574b7ac1f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:10:35.943472 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c668ddcf-4df5-4f8a-9e6d-86574b7ac1f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:10:35.943749 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c668ddcf-4df5-4f8a-9e6d-86574b7ac1f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:10:35.944021 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c668ddcf-4df5-4f8a-9e6d-86574b7ac1f1 admin admin] PATH_INFO: `/v3/projects/a5ab4a0a80954a34a11b0db3ea81b14f` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:10:35.953849 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c668ddcf-4df5-4f8a-9e6d-86574b7ac1f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5ab4a0a80954a34a11b0db3ea81b14f)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:10:35.954861 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: WARNING py.warnings [None req-c668ddcf-4df5-4f8a-9e6d-86574b7ac1f1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 12:10:35.954861 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: warnings.warn(msg) Aug 26 12:10:35.954861 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: Aug 26 12:10:35.956091 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c668ddcf-4df5-4f8a-9e6d-86574b7ac1f1 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:10:36.015917 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-c668ddcf-4df5-4f8a-9e6d-86574b7ac1f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5ab4a0a80954a34a11b0db3ea81b14f', 'request_id': 'req-c668ddcf-4df5-4f8a-9e6d-86574b7ac1f1'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:10:36.029317 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-c668ddcf-4df5-4f8a-9e6d-86574b7ac1f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5ab4a0a80954a34a11b0db3ea81b14f', 'request_id': 'req-c668ddcf-4df5-4f8a-9e6d-86574b7ac1f1'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:10:36.038523 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-c668ddcf-4df5-4f8a-9e6d-86574b7ac1f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5ab4a0a80954a34a11b0db3ea81b14f', 'request_id': 'req-c668ddcf-4df5-4f8a-9e6d-86574b7ac1f1'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:10:36.061967 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-c668ddcf-4df5-4f8a-9e6d-86574b7ac1f1 admin admin] The token cache is being invalidate because project a5ab4a0a80954a34a11b0db3ea81b14f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81712) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:10:36.062256 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-c668ddcf-4df5-4f8a-9e6d-86574b7ac1f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:10:36.062684 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-c668ddcf-4df5-4f8a-9e6d-86574b7ac1f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:10:36.064119 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 759/1510] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:10:35 2022] DELETE /identity/v3/projects/a5ab4a0a80954a34a11b0db3ea81b14f => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:10:45.477268 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=62 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:10:46.059919 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:12:04.946687 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad7d87d-1a1e-4106-89bd-d1392d92532c None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:12:04.947303 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad7d87d-1a1e-4106-89bd-d1392d92532c None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:12:04.948039 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad7d87d-1a1e-4106-89bd-d1392d92532c None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:12:04.949659 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 752/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 12:12:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:12:04.955898 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d91234-cc4c-4d0b-a3f3-4029320fcae4 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:12:04.956527 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d91234-cc4c-4d0b-a3f3-4029320fcae4 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:12:04.956811 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d91234-cc4c-4d0b-a3f3-4029320fcae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:12:05.041503 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-07d91234-cc4c-4d0b-a3f3-4029320fcae4 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:12:05.042649 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-07d91234-cc4c-4d0b-a3f3-4029320fcae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:12:05.145120 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 760/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 12:12:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:12:05.300592 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad7d87d-1a1e-4106-89bd-d1392d92532c None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:12:05.302779 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-0ad7d87d-1a1e-4106-89bd-d1392d92532c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:12:05.419766 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad7d87d-1a1e-4106-89bd-d1392d92532c None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:12:05.425094 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7024b3a-e660-4786-8087-f6d0517a9fc3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aacd7d6d17e4f10ba5c9107a12beafa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:12:05.425685 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7024b3a-e660-4786-8087-f6d0517a9fc3 service nova] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:12:05.425968 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7024b3a-e660-4786-8087-f6d0517a9fc3 service nova] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:12:05.426240 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7024b3a-e660-4786-8087-f6d0517a9fc3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:12:05.434921 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7024b3a-e660-4786-8087-f6d0517a9fc3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:12:05.436596 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7024b3a-e660-4786-8087-f6d0517a9fc3 service nova] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:12:05.445717 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 753/1513] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:12:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:12:06.268704 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3828be62-e78a-4db8-a733-5b9edbf77ba4 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:12:06.269319 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3828be62-e78a-4db8-a733-5b9edbf77ba4 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:12:06.269319 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3828be62-e78a-4db8-a733-5b9edbf77ba4 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:12:06.270670 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 761/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 12:12:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:12:06.275880 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a399d3c-7d58-471f-bb2b-ab67af91a253 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:12:06.276356 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a399d3c-7d58-471f-bb2b-ab67af91a253 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:12:06.276464 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a399d3c-7d58-471f-bb2b-ab67af91a253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:12:06.295795 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-3a399d3c-7d58-471f-bb2b-ab67af91a253 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:12:06.296965 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-3a399d3c-7d58-471f-bb2b-ab67af91a253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:12:06.304603 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 754/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 12:12:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:12:06.459129 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3828be62-e78a-4db8-a733-5b9edbf77ba4 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:12:06.473971 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3828be62-e78a-4db8-a733-5b9edbf77ba4 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:12:06.479441 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3570c886-5b36-42f6-ae83-e07e33b810ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aacd7d6d17e4f10ba5c9107a12beafa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:12:06.480027 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3570c886-5b36-42f6-ae83-e07e33b810ba service nova] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:12:06.480306 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3570c886-5b36-42f6-ae83-e07e33b810ba service nova] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:12:06.480578 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3570c886-5b36-42f6-ae83-e07e33b810ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:12:06.488627 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3570c886-5b36-42f6-ae83-e07e33b810ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:12:06.490203 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3570c886-5b36-42f6-ae83-e07e33b810ba service nova] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:12:06.499889 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 762/1516] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:12:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:12:07.097454 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838c6744-2bfa-483f-a33a-59b1e44fa1d3 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:12:07.097739 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838c6744-2bfa-483f-a33a-59b1e44fa1d3 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:12:07.098011 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838c6744-2bfa-483f-a33a-59b1e44fa1d3 None None] PATH_INFO: `/v3` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:12:07.099388 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 755/1517] 10.222.0.30 () {54 vars in 925 bytes} [Fri Aug 26 12:12:07 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:12:07.125090 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5534141c-d322-419b-aa49-5e088f385722 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:12:07.125314 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5534141c-d322-419b-aa49-5e088f385722 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:12:07.125585 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5534141c-d322-419b-aa49-5e088f385722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:12:07.184854 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-5534141c-d322-419b-aa49-5e088f385722 None None] MFA Rules not processed for user `7346e4a29dfb4a0a9768597704be286c`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:12:07.186094 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-5534141c-d322-419b-aa49-5e088f385722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:12:07.202887 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:12:07.258346 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 763/1518] 10.222.0.30 () {58 vars in 1015 bytes} [Fri Aug 26 12:12:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:12:11.117130 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faee8b3-fccc-476f-8cb7-7a1363b89c94 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:12:11.117648 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faee8b3-fccc-476f-8cb7-7a1363b89c94 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:12:11.117648 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faee8b3-fccc-476f-8cb7-7a1363b89c94 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:12:11.119022 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 756/1519] 10.222.0.61 () {54 vars in 961 bytes} [Fri Aug 26 12:12:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:12:11.124381 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7bda47-394f-4527-a8c1-0e9684cf9226 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:12:11.124786 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7bda47-394f-4527-a8c1-0e9684cf9226 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:12:11.132264 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7bda47-394f-4527-a8c1-0e9684cf9226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:12:11.154323 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-cb7bda47-394f-4527-a8c1-0e9684cf9226 None None] MFA Rules not processed for user `7346e4a29dfb4a0a9768597704be286c`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:12:11.155532 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-cb7bda47-394f-4527-a8c1-0e9684cf9226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:12:11.163874 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 764/1520] 10.222.0.61 () {58 vars in 1058 bytes} [Fri Aug 26 12:12:11 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:12:11.171578 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faee8b3-fccc-476f-8cb7-7a1363b89c94 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:12:11.184705 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faee8b3-fccc-476f-8cb7-7a1363b89c94 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:12:11.190499 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd56e954-9521-44c4-9940-8c936e3b9c2a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7346e4a29dfb4a0a9768597704be286c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:12:11.191085 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd56e954-9521-44c4-9940-8c936e3b9c2a service placement] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:12:11.191394 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd56e954-9521-44c4-9940-8c936e3b9c2a service placement] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:12:11.191667 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd56e954-9521-44c4-9940-8c936e3b9c2a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:12:11.200528 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd56e954-9521-44c4-9940-8c936e3b9c2a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:12:11.202087 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd56e954-9521-44c4-9940-8c936e3b9c2a service placement] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:12:11.211425 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 757/1521] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:12:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:12:17.729259 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460bb5d5-1fdc-4ace-b2fb-0d5caad508ae None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:12:17.729823 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460bb5d5-1fdc-4ace-b2fb-0d5caad508ae None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:12:17.729823 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460bb5d5-1fdc-4ace-b2fb-0d5caad508ae None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:12:17.731198 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 765/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 12:12:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:12:17.736035 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b1e24-a1d5-48f8-b33e-fedf1a5b5fa0 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:12:17.736459 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b1e24-a1d5-48f8-b33e-fedf1a5b5fa0 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:12:17.736582 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b1e24-a1d5-48f8-b33e-fedf1a5b5fa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:12:17.757016 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-e65b1e24-a1d5-48f8-b33e-fedf1a5b5fa0 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:12:17.758193 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-e65b1e24-a1d5-48f8-b33e-fedf1a5b5fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:12:17.766072 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 758/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 12:12:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:12:17.919450 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460bb5d5-1fdc-4ace-b2fb-0d5caad508ae None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:12:17.934683 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460bb5d5-1fdc-4ace-b2fb-0d5caad508ae None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:12:17.940625 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee7f1d2-a27c-42d9-ba1a-b2de880cab6c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aacd7d6d17e4f10ba5c9107a12beafa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:12:17.941222 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee7f1d2-a27c-42d9-ba1a-b2de880cab6c service nova] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:12:17.941498 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee7f1d2-a27c-42d9-ba1a-b2de880cab6c service nova] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:12:17.941769 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee7f1d2-a27c-42d9-ba1a-b2de880cab6c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:12:17.950253 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee7f1d2-a27c-42d9-ba1a-b2de880cab6c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:12:17.951992 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee7f1d2-a27c-42d9-ba1a-b2de880cab6c service nova] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:12:17.961410 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 766/1524] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:12:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:12:18.780583 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1d3a1-8197-4ed1-bb64-91a6605eb398 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:12:18.781082 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1d3a1-8197-4ed1-bb64-91a6605eb398 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:12:18.781179 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d1d3a1-8197-4ed1-bb64-91a6605eb398 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:12:18.782459 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 759/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 26 12:12:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:12:18.787284 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9cfc95-7321-433f-a745-ff6cf9155ac8 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:12:18.787695 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9cfc95-7321-433f-a745-ff6cf9155ac8 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:12:18.787831 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9cfc95-7321-433f-a745-ff6cf9155ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:12:18.807642 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-0e9cfc95-7321-433f-a745-ff6cf9155ac8 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:12:18.808815 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-0e9cfc95-7321-433f-a745-ff6cf9155ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:12:18.817014 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 767/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 26 12:12:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:12:18.968960 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d1d3a1-8197-4ed1-bb64-91a6605eb398 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:12:18.983060 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d1d3a1-8197-4ed1-bb64-91a6605eb398 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:12:18.988389 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d11c191-8596-4a78-aae1-4ec0e7f42a62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aacd7d6d17e4f10ba5c9107a12beafa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:12:18.988974 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d11c191-8596-4a78-aae1-4ec0e7f42a62 service nova] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:12:18.989252 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d11c191-8596-4a78-aae1-4ec0e7f42a62 service nova] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:12:18.989522 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d11c191-8596-4a78-aae1-4ec0e7f42a62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:12:18.998385 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d11c191-8596-4a78-aae1-4ec0e7f42a62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:12:18.999932 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d11c191-8596-4a78-aae1-4ec0e7f42a62 service nova] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:12:19.009205 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 760/1527] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:12:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:12:22.603784 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95da73c-0f72-4637-b4a2-41d3d1666a98 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:12:22.604324 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95da73c-0f72-4637-b4a2-41d3d1666a98 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:12:22.604324 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95da73c-0f72-4637-b4a2-41d3d1666a98 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:12:22.605670 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 768/1528] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 12:12:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:12:22.610940 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11e4946-1989-4035-a86f-d72559ac83cf None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:12:22.611395 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11e4946-1989-4035-a86f-d72559ac83cf None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:12:22.611530 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11e4946-1989-4035-a86f-d72559ac83cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:12:22.631010 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-c11e4946-1989-4035-a86f-d72559ac83cf None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:12:22.632216 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-c11e4946-1989-4035-a86f-d72559ac83cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:12:22.640573 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 761/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 12:12:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:12:22.928155 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95da73c-0f72-4637-b4a2-41d3d1666a98 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:12:22.929897 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-c95da73c-0f72-4637-b4a2-41d3d1666a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:12:23.008938 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95da73c-0f72-4637-b4a2-41d3d1666a98 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:12:23.012479 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102bea9a-ae57-49c2-805e-a286e51628e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:12:23.013075 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102bea9a-ae57-49c2-805e-a286e51628e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:12:23.013351 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102bea9a-ae57-49c2-805e-a286e51628e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:12:23.013623 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102bea9a-ae57-49c2-805e-a286e51628e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:12:23.022337 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102bea9a-ae57-49c2-805e-a286e51628e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:12:23.023810 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102bea9a-ae57-49c2-805e-a286e51628e2 service neutron] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:12:23.033438 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 769/1530] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:12:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:12:24.932560 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774a7f34-2544-440f-b6f0-5b896532a10a None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:12:24.933112 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774a7f34-2544-440f-b6f0-5b896532a10a None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:12:24.933112 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774a7f34-2544-440f-b6f0-5b896532a10a None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:12:24.934532 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 762/1531] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 12:12:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:12:24.939629 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e4de1-7900-450d-91fe-cd3057866743 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:12:24.940030 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e4de1-7900-450d-91fe-cd3057866743 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:12:24.940166 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e4de1-7900-450d-91fe-cd3057866743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:12:24.959004 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-471e4de1-7900-450d-91fe-cd3057866743 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:12:24.960209 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-471e4de1-7900-450d-91fe-cd3057866743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:12:24.968390 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 770/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 12:12:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:12:25.251227 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774a7f34-2544-440f-b6f0-5b896532a10a None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:12:25.262420 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774a7f34-2544-440f-b6f0-5b896532a10a None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:12:25.265845 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5463d7-c9c6-4d16-88c1-4d67d5bc7162 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:12:25.266429 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5463d7-c9c6-4d16-88c1-4d67d5bc7162 service neutron] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:12:25.266732 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5463d7-c9c6-4d16-88c1-4d67d5bc7162 service neutron] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:12:25.266999 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5463d7-c9c6-4d16-88c1-4d67d5bc7162 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:12:25.276275 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5463d7-c9c6-4d16-88c1-4d67d5bc7162 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:12:25.277684 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5463d7-c9c6-4d16-88c1-4d67d5bc7162 service neutron] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:12:25.286797 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 763/1533] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:12:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:12:28.847107 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27a62ec-8d52-4cb7-b98e-87f921d98724 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:12:28.847695 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27a62ec-8d52-4cb7-b98e-87f921d98724 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:12:28.847695 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27a62ec-8d52-4cb7-b98e-87f921d98724 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:12:28.849025 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 771/1534] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 12:12:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:12:28.854524 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d99d7b-98bc-4d7a-8123-ca0c4fcb4d2d None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:12:28.854952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d99d7b-98bc-4d7a-8123-ca0c4fcb4d2d None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:12:28.855065 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d99d7b-98bc-4d7a-8123-ca0c4fcb4d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:12:28.862607 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=81 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:12:28.875988 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-94d99d7b-98bc-4d7a-8123-ca0c4fcb4d2d None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:12:28.877174 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-94d99d7b-98bc-4d7a-8123-ca0c4fcb4d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:12:28.885749 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 764/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 12:12:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:12:29.458939 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27a62ec-8d52-4cb7-b98e-87f921d98724 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:12:29.460627 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-a27a62ec-8d52-4cb7-b98e-87f921d98724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:12:29.526018 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:12:29.541089 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27a62ec-8d52-4cb7-b98e-87f921d98724 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:12:29.546335 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e429914-8f46-4a0c-9128-c898f5f8090a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:12:29.546947 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e429914-8f46-4a0c-9128-c898f5f8090a service cinder] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:12:29.547249 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e429914-8f46-4a0c-9128-c898f5f8090a service cinder] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:12:29.547525 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e429914-8f46-4a0c-9128-c898f5f8090a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:12:29.556666 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e429914-8f46-4a0c-9128-c898f5f8090a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:12:29.558089 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e429914-8f46-4a0c-9128-c898f5f8090a service cinder] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:12:29.567251 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 772/1536] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:12:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:12:31.485688 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4fbe80-86e4-4e01-aa81-32246c18783c None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:12:31.486230 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4fbe80-86e4-4e01-aa81-32246c18783c None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:12:31.486230 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4fbe80-86e4-4e01-aa81-32246c18783c None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:12:31.487609 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 765/1537] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 26 12:12:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:12:31.492721 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2651b6a-7f83-4bb0-a305-db62cdfe1bef None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:12:31.493166 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2651b6a-7f83-4bb0-a305-db62cdfe1bef None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:12:31.493300 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2651b6a-7f83-4bb0-a305-db62cdfe1bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:12:31.513406 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-c2651b6a-7f83-4bb0-a305-db62cdfe1bef None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:12:31.514592 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-c2651b6a-7f83-4bb0-a305-db62cdfe1bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:12:31.522409 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 773/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 26 12:12:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:12:32.072409 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4fbe80-86e4-4e01-aa81-32246c18783c None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:12:32.074094 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-1d4fbe80-86e4-4e01-aa81-32246c18783c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:12:32.086788 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4fbe80-86e4-4e01-aa81-32246c18783c None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:12:32.090829 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cc390a-0861-4d41-b22b-61a3a488c715 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:12:32.091442 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc390a-0861-4d41-b22b-61a3a488c715 service cinder] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:12:32.091723 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc390a-0861-4d41-b22b-61a3a488c715 service cinder] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:12:32.091993 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc390a-0861-4d41-b22b-61a3a488c715 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:12:32.100649 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cc390a-0861-4d41-b22b-61a3a488c715 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:12:32.102057 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cc390a-0861-4d41-b22b-61a3a488c715 service cinder] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:12:32.111330 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 766/1539] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:12:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:12:41.508166 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=11 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:12:42.107205 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=8 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:13:57.519442 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b6899-9ef2-4e06-8044-404a310825e9 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:57.523723 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b6899-9ef2-4e06-8044-404a310825e9 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:57.523723 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b6899-9ef2-4e06-8044-404a310825e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:57.544788 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-641b6899-9ef2-4e06-8044-404a310825e9 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:13:57.545987 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-641b6899-9ef2-4e06-8044-404a310825e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:13:57.560073 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 774/1540] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:13:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:13:57.568831 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b94261-b589-4e93-b03d-eed362dc8c4f None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:57.572435 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b94261-b589-4e93-b03d-eed362dc8c4f None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:57.572435 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b94261-b589-4e93-b03d-eed362dc8c4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:57.607683 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-98b94261-b589-4e93-b03d-eed362dc8c4f None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:13:57.608960 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-98b94261-b589-4e93-b03d-eed362dc8c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:13:57.623886 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 767/1541] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:13:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:13:57.628845 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641b6899-9ef2-4e06-8044-404a310825e9 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:13:57.645562 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641b6899-9ef2-4e06-8044-404a310825e9 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:13:57.655005 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f63307-04ac-4d45-9838-3004bccce7a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:13:57.655671 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f63307-04ac-4d45-9838-3004bccce7a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:57.655951 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f63307-04ac-4d45-9838-3004bccce7a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:57.656225 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f63307-04ac-4d45-9838-3004bccce7a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:57.656899 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f63307-04ac-4d45-9838-3004bccce7a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:13:57.657204 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f63307-04ac-4d45-9838-3004bccce7a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:13:57.658874 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f63307-04ac-4d45-9838-3004bccce7a3 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:13:57.678572 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 775/1542] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:13:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:13:57.687130 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b94261-b589-4e93-b03d-eed362dc8c4f None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:13:57.700416 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b94261-b589-4e93-b03d-eed362dc8c4f None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:13:57.707864 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee491fa7-6802-452e-ae1b-c9beca844a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:13:57.708486 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee491fa7-6802-452e-ae1b-c9beca844a6a admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:57.708816 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee491fa7-6802-452e-ae1b-c9beca844a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:57.709094 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee491fa7-6802-452e-ae1b-c9beca844a6a admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:57.709727 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee491fa7-6802-452e-ae1b-c9beca844a6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:13:57.711837 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee491fa7-6802-452e-ae1b-c9beca844a6a admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:13:57.786957 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 768/1543] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:13:57 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:13:57.791495 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f63307-04ac-4d45-9838-3004bccce7a3 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:13:57.804513 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f63307-04ac-4d45-9838-3004bccce7a3 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:13:57.885457 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70738de-94e3-46b8-ae40-71a1beddc3c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:13:57.886076 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70738de-94e3-46b8-ae40-71a1beddc3c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:57.886359 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70738de-94e3-46b8-ae40-71a1beddc3c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:57.886632 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70738de-94e3-46b8-ae40-71a1beddc3c8 admin admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:57.887309 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70738de-94e3-46b8-ae40-71a1beddc3c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:13:57.889282 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70738de-94e3-46b8-ae40-71a1beddc3c8 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:13:57.927914 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 776/1544] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:13:57 2022] POST /identity/v3/users => generated 293 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:13:57.936713 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee491fa7-6802-452e-ae1b-c9beca844a6a admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:13:57.949507 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee491fa7-6802-452e-ae1b-c9beca844a6a admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:13:57.958460 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117396de-fa28-4a34-842b-c4e3cc58b799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:13:57.959089 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117396de-fa28-4a34-842b-c4e3cc58b799 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:57.959425 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117396de-fa28-4a34-842b-c4e3cc58b799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:57.959750 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117396de-fa28-4a34-842b-c4e3cc58b799 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:57.960443 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117396de-fa28-4a34-842b-c4e3cc58b799 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:13:57.962049 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117396de-fa28-4a34-842b-c4e3cc58b799 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:13:57.981779 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 769/1545] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:13:57 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:13:57.986441 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70738de-94e3-46b8-ae40-71a1beddc3c8 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:13:57.998738 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70738de-94e3-46b8-ae40-71a1beddc3c8 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:13:58.008869 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fca867-4d98-4b7a-9ad7-7f281cb7a6da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:13:58.009534 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fca867-4d98-4b7a-9ad7-7f281cb7a6da admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:58.009813 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fca867-4d98-4b7a-9ad7-7f281cb7a6da admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:58.010271 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fca867-4d98-4b7a-9ad7-7f281cb7a6da admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:58.010921 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fca867-4d98-4b7a-9ad7-7f281cb7a6da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:13:58.012658 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fca867-4d98-4b7a-9ad7-7f281cb7a6da admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:13:58.031007 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 777/1546] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:13:57 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:13:58.035462 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117396de-fa28-4a34-842b-c4e3cc58b799 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:13:58.049411 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117396de-fa28-4a34-842b-c4e3cc58b799 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:13:58.056451 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f8ce8-b12f-4c32-9528-2c1c7dd637ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:13:58.057126 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f8ce8-b12f-4c32-9528-2c1c7dd637ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:58.057405 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f8ce8-b12f-4c32-9528-2c1c7dd637ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:58.057680 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f8ce8-b12f-4c32-9528-2c1c7dd637ce admin admin] PATH_INFO: `/v3/projects/4b937d8a156b4be0994719f3399f2d14/users/77ba5afd671e4b2b95470d20c170651e/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:58.065739 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5631d6a-41a3-4f12-825d-7027a65cd222 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:58.066051 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5631d6a-41a3-4f12-825d-7027a65cd222 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:58.066296 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5631d6a-41a3-4f12-825d-7027a65cd222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:58.089894 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2f8ce8-b12f-4c32-9528-2c1c7dd637ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b937d8a156b4be0994719f3399f2d14, user_id=77ba5afd671e4b2b95470d20c170651e, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:13:58.099455 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2f8ce8-b12f-4c32-9528-2c1c7dd637ce admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:13:58.115232 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-d5631d6a-41a3-4f12-825d-7027a65cd222 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:13:58.116627 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-d5631d6a-41a3-4f12-825d-7027a65cd222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:13:58.130434 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 770/1547] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:13:58 2022] PUT /identity/v3/projects/4b937d8a156b4be0994719f3399f2d14/users/77ba5afd671e4b2b95470d20c170651e/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:13:58.131741 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac0b97c-daaf-43b7-a2e2-c0a469262640 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:58.132790 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac0b97c-daaf-43b7-a2e2-c0a469262640 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:58.133313 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac0b97c-daaf-43b7-a2e2-c0a469262640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:58.133533 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 778/1548] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:13:58.135494 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0c31c-3c8c-43f8-b83b-37d59b3d9f92 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:58.135919 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0c31c-3c8c-43f8-b83b-37d59b3d9f92 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:58.136246 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0c31c-3c8c-43f8-b83b-37d59b3d9f92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:58.160164 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-2ae0c31c-3c8c-43f8-b83b-37d59b3d9f92 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:13:58.161365 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-2ae0c31c-3c8c-43f8-b83b-37d59b3d9f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:13:58.167663 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-2ac0b97c-daaf-43b7-a2e2-c0a469262640 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:13:58.169688 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-2ac0b97c-daaf-43b7-a2e2-c0a469262640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:13:58.231391 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 779/1549] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:13:58.232681 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d44cb-6442-4295-870e-1953ddc52a75 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:58.232968 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d44cb-6442-4295-870e-1953ddc52a75 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:58.233240 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d44cb-6442-4295-870e-1953ddc52a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:58.243522 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 771/1550] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:13:58.245185 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa97d5c-d9de-4ca7-a5b4-ecc736ad4c76 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:58.245473 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa97d5c-d9de-4ca7-a5b4-ecc736ad4c76 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:58.245750 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa97d5c-d9de-4ca7-a5b4-ecc736ad4c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:58.254535 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-df1d44cb-6442-4295-870e-1953ddc52a75 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:13:58.255789 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-df1d44cb-6442-4295-870e-1953ddc52a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:13:58.263904 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 780/1551] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:13:58.271253 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-ffa97d5c-d9de-4ca7-a5b4-ecc736ad4c76 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:13:58.271380 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eb7d95-126c-4736-829b-056bd8a9ad79 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:58.271380 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eb7d95-126c-4736-829b-056bd8a9ad79 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:58.271585 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-ffa97d5c-d9de-4ca7-a5b4-ecc736ad4c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:13:58.271675 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eb7d95-126c-4736-829b-056bd8a9ad79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:58.275886 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 772/1552] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:13:58.277090 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c458f4d6-606d-4691-b248-6a34ecd9bcbe None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:58.277365 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c458f4d6-606d-4691-b248-6a34ecd9bcbe None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:58.277659 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c458f4d6-606d-4691-b248-6a34ecd9bcbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:58.285554 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-d2eb7d95-126c-4736-829b-056bd8a9ad79 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:13:58.286841 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-d2eb7d95-126c-4736-829b-056bd8a9ad79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:13:58.296773 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 781/1553] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:13:58.297986 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4165ec8-1324-47b1-8196-0df0539dd2b1 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:58.298264 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4165ec8-1324-47b1-8196-0df0539dd2b1 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:58.298535 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4165ec8-1324-47b1-8196-0df0539dd2b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:58.314288 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-c458f4d6-606d-4691-b248-6a34ecd9bcbe None None] MFA Rules not processed for user `77ba5afd671e4b2b95470d20c170651e`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:13:58.315436 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-c458f4d6-606d-4691-b248-6a34ecd9bcbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:13:58.318215 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-f4165ec8-1324-47b1-8196-0df0539dd2b1 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:13:58.319401 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-f4165ec8-1324-47b1-8196-0df0539dd2b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:13:58.327796 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 782/1554] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:13:58.328962 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317a7ca1-92dd-4d45-8612-a82b2ae80074 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:58.329241 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317a7ca1-92dd-4d45-8612-a82b2ae80074 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:58.329507 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317a7ca1-92dd-4d45-8612-a82b2ae80074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:58.348624 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-317a7ca1-92dd-4d45-8612-a82b2ae80074 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:13:58.349799 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-317a7ca1-92dd-4d45-8612-a82b2ae80074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:13:58.358447 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 783/1555] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:13:58.360539 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b547567a-4160-4391-a237-610689140aff None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:58.360826 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b547567a-4160-4391-a237-610689140aff None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:58.361108 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b547567a-4160-4391-a237-610689140aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:58.381399 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-b547567a-4160-4391-a237-610689140aff None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:13:58.382587 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-b547567a-4160-4391-a237-610689140aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:13:58.389151 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 773/1556] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:13:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:13:58.390411 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d44e6c4-34c3-4194-8dda-3d0fd5279807 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:58.394509 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d44e6c4-34c3-4194-8dda-3d0fd5279807 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:58.394654 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 784/1557] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:13:58.394741 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d44e6c4-34c3-4194-8dda-3d0fd5279807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:58.394827 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db68c4b-bb9f-477b-b61e-415daf488dc0 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:58.394827 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db68c4b-bb9f-477b-b61e-415daf488dc0 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:58.394827 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db68c4b-bb9f-477b-b61e-415daf488dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:58.410327 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-2d44e6c4-34c3-4194-8dda-3d0fd5279807 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:13:58.411063 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-2db68c4b-bb9f-477b-b61e-415daf488dc0 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:13:58.411553 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-2d44e6c4-34c3-4194-8dda-3d0fd5279807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:13:58.412236 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-2db68c4b-bb9f-477b-b61e-415daf488dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:13:58.420044 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 774/1558] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:13:58.421345 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1831a5-eb4b-457c-8046-0a05c81946f7 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:58.421664 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1831a5-eb4b-457c-8046-0a05c81946f7 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:58.421939 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1831a5-eb4b-457c-8046-0a05c81946f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:58.424305 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 785/1559] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:13:58.424936 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db68c4b-bb9f-477b-b61e-415daf488dc0 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:13:58.431853 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:13:58.435850 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db68c4b-bb9f-477b-b61e-415daf488dc0 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:13:58.440640 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed256d13-8300-497c-9d4c-3a86fd166645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:13:58.441243 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed256d13-8300-497c-9d4c-3a86fd166645 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:58.441519 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed256d13-8300-497c-9d4c-3a86fd166645 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:58.441797 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed256d13-8300-497c-9d4c-3a86fd166645 admin admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:58.442455 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed256d13-8300-497c-9d4c-3a86fd166645 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:13:58.442761 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed256d13-8300-497c-9d4c-3a86fd166645 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:13:58.443849 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-be1831a5-eb4b-457c-8046-0a05c81946f7 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:13:58.444425 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed256d13-8300-497c-9d4c-3a86fd166645 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:13:58.445140 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-be1831a5-eb4b-457c-8046-0a05c81946f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:13:58.453154 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:13:58.454357 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 775/1560] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:13:58.455587 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b455dcd-91c1-40f0-917a-988cf62bb331 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:58.455867 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b455dcd-91c1-40f0-917a-988cf62bb331 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:58.456161 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b455dcd-91c1-40f0-917a-988cf62bb331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:58.461217 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 786/1561] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:13:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:13:58.468245 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c558f7e2-2912-4871-b2b7-7542f5eeb5dc None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:58.468245 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c558f7e2-2912-4871-b2b7-7542f5eeb5dc None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:58.468245 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c558f7e2-2912-4871-b2b7-7542f5eeb5dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:58.476325 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-2b455dcd-91c1-40f0-917a-988cf62bb331 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:13:58.477522 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-2b455dcd-91c1-40f0-917a-988cf62bb331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:13:58.484402 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-c558f7e2-2912-4871-b2b7-7542f5eeb5dc None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:13:58.485544 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-c558f7e2-2912-4871-b2b7-7542f5eeb5dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:13:58.486098 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 776/1562] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:13:58.486850 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b455dcd-91c1-40f0-917a-988cf62bb331 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:13:58.494141 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 787/1563] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:13:58.495699 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c558f7e2-2912-4871-b2b7-7542f5eeb5dc None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:13:58.496375 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b455dcd-91c1-40f0-917a-988cf62bb331 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:13:58.500971 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef65bc4-a901-4f12-8e0e-aa1d19000bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:13:58.501561 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef65bc4-a901-4f12-8e0e-aa1d19000bc4 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:58.501841 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef65bc4-a901-4f12-8e0e-aa1d19000bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:58.502113 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef65bc4-a901-4f12-8e0e-aa1d19000bc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:58.502758 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef65bc4-a901-4f12-8e0e-aa1d19000bc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:13:58.504684 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef65bc4-a901-4f12-8e0e-aa1d19000bc4 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:13:58.505341 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c558f7e2-2912-4871-b2b7-7542f5eeb5dc None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:13:58.510306 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ee934c-e215-4e44-a8ba-2cf3d519bbc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:13:58.510892 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee934c-e215-4e44-a8ba-2cf3d519bbc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:58.511171 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee934c-e215-4e44-a8ba-2cf3d519bbc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:58.511481 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee934c-e215-4e44-a8ba-2cf3d519bbc7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:58.512116 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ee934c-e215-4e44-a8ba-2cf3d519bbc7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:13:58.512415 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ee934c-e215-4e44-a8ba-2cf3d519bbc7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:13:58.513998 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ee934c-e215-4e44-a8ba-2cf3d519bbc7 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:13:58.527612 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 788/1564] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:13:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:13:58.528438 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ee934c-e215-4e44-a8ba-2cf3d519bbc7 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:13:58.532936 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 777/1565] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:13:58 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:13:58.536451 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef65bc4-a901-4f12-8e0e-aa1d19000bc4 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:13:58.541659 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ee934c-e215-4e44-a8ba-2cf3d519bbc7 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:13:58.596832 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b2835f-21c6-4c61-9698-e264fc07c432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:13:58.597352 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b2835f-21c6-4c61-9698-e264fc07c432 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:58.597630 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b2835f-21c6-4c61-9698-e264fc07c432 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:58.597903 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b2835f-21c6-4c61-9698-e264fc07c432 admin admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:58.598127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef65bc4-a901-4f12-8e0e-aa1d19000bc4 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:13:58.598594 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b2835f-21c6-4c61-9698-e264fc07c432 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:13:58.598826 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b2835f-21c6-4c61-9698-e264fc07c432 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:13:58.600456 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b2835f-21c6-4c61-9698-e264fc07c432 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:13:58.603596 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9970c33-57d0-4a6f-8bbf-4a09c9e8d1af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:13:58.604204 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9970c33-57d0-4a6f-8bbf-4a09c9e8d1af admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:58.604483 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9970c33-57d0-4a6f-8bbf-4a09c9e8d1af admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:58.604759 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9970c33-57d0-4a6f-8bbf-4a09c9e8d1af admin admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:58.605403 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9970c33-57d0-4a6f-8bbf-4a09c9e8d1af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:13:58.605706 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9970c33-57d0-4a6f-8bbf-4a09c9e8d1af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:13:58.606718 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: WARNING py.warnings [None req-c9970c33-57d0-4a6f-8bbf-4a09c9e8d1af admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 26 12:13:58.606718 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: warnings.warn(msg) Aug 26 12:13:58.606718 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: Aug 26 12:13:58.607632 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9970c33-57d0-4a6f-8bbf-4a09c9e8d1af admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:13:58.614269 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 789/1566] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:13:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:13:58.615062 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b2835f-21c6-4c61-9698-e264fc07c432 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:13:58.620814 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 778/1567] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:13:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:13:58.621626 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9970c33-57d0-4a6f-8bbf-4a09c9e8d1af admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:13:58.625980 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b2835f-21c6-4c61-9698-e264fc07c432 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:13:58.631756 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca5101-9dfb-44a2-8305-d22fead2c07c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:13:58.631937 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9970c33-57d0-4a6f-8bbf-4a09c9e8d1af admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:13:58.632354 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca5101-9dfb-44a2-8305-d22fead2c07c admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:58.632632 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca5101-9dfb-44a2-8305-d22fead2c07c admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:58.632903 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca5101-9dfb-44a2-8305-d22fead2c07c admin admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:58.633533 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ca5101-9dfb-44a2-8305-d22fead2c07c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:13:58.633858 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ca5101-9dfb-44a2-8305-d22fead2c07c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:13:58.636186 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ca5101-9dfb-44a2-8305-d22fead2c07c admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:13:58.637588 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c1df05-8c20-404d-abbd-1dfb72be032c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:13:58.638179 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c1df05-8c20-404d-abbd-1dfb72be032c admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:58.638457 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c1df05-8c20-404d-abbd-1dfb72be032c admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:58.638729 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c1df05-8c20-404d-abbd-1dfb72be032c admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:58.639373 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c1df05-8c20-404d-abbd-1dfb72be032c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:13:58.641252 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c1df05-8c20-404d-abbd-1dfb72be032c admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:13:58.648008 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 790/1568] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:13:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:13:58.652537 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca5101-9dfb-44a2-8305-d22fead2c07c admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:13:58.663103 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca5101-9dfb-44a2-8305-d22fead2c07c admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:13:58.668429 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e668c50a-b09b-4c38-9c4b-269488ed1f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:13:58.669023 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e668c50a-b09b-4c38-9c4b-269488ed1f01 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:58.669305 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e668c50a-b09b-4c38-9c4b-269488ed1f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:58.669579 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e668c50a-b09b-4c38-9c4b-269488ed1f01 admin admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:58.670218 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e668c50a-b09b-4c38-9c4b-269488ed1f01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:13:58.670523 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e668c50a-b09b-4c38-9c4b-269488ed1f01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:13:58.671417 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 779/1569] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:13:58 2022] POST /identity/v3/projects => generated 403 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:13:58.672185 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e668c50a-b09b-4c38-9c4b-269488ed1f01 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:13:58.672313 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c1df05-8c20-404d-abbd-1dfb72be032c admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:13:58.685678 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 791/1570] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:13:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:13:58.688088 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e668c50a-b09b-4c38-9c4b-269488ed1f01 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:13:58.699982 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e668c50a-b09b-4c38-9c4b-269488ed1f01 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:13:58.735579 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c1df05-8c20-404d-abbd-1dfb72be032c admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:13:58.745225 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa228002-516e-46f8-aa39-9f4cb1fa42b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:13:58.746648 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa228002-516e-46f8-aa39-9f4cb1fa42b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:58.747103 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa228002-516e-46f8-aa39-9f4cb1fa42b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:58.747419 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa228002-516e-46f8-aa39-9f4cb1fa42b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:58.748414 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa228002-516e-46f8-aa39-9f4cb1fa42b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:13:58.748755 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa228002-516e-46f8-aa39-9f4cb1fa42b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:13:58.750999 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa228002-516e-46f8-aa39-9f4cb1fa42b3 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:13:58.759056 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16d0e56-0061-48d3-bbce-bb5f26057b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:13:58.760372 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16d0e56-0061-48d3-bbce-bb5f26057b5a admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:58.760663 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16d0e56-0061-48d3-bbce-bb5f26057b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:58.760935 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16d0e56-0061-48d3-bbce-bb5f26057b5a admin admin] PATH_INFO: `/v3/projects` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:58.761565 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16d0e56-0061-48d3-bbce-bb5f26057b5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:13:58.762576 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: WARNING py.warnings [None req-a16d0e56-0061-48d3-bbce-bb5f26057b5a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 12:13:58.762576 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: warnings.warn(msg) Aug 26 12:13:58.762576 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: Aug 26 12:13:58.763815 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16d0e56-0061-48d3-bbce-bb5f26057b5a admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:13:58.764077 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 780/1571] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:13:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:13:58.771253 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa228002-516e-46f8-aa39-9f4cb1fa42b3 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:13:58.776259 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa228002-516e-46f8-aa39-9f4cb1fa42b3 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:13:58.780833 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c87d76e-336c-457b-a110-734c65be71fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:13:58.781536 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c87d76e-336c-457b-a110-734c65be71fc admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:58.781907 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c87d76e-336c-457b-a110-734c65be71fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:58.782187 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c87d76e-336c-457b-a110-734c65be71fc admin admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:58.782994 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c87d76e-336c-457b-a110-734c65be71fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:13:58.784047 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: WARNING py.warnings [None req-5c87d76e-336c-457b-a110-734c65be71fc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 12:13:58.784047 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: warnings.warn(msg) Aug 26 12:13:58.784047 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: Aug 26 12:13:58.785373 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c87d76e-336c-457b-a110-734c65be71fc admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:13:58.792063 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 792/1572] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:13:58 2022] POST /identity/v3/projects => generated 391 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:13:58.792873 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16d0e56-0061-48d3-bbce-bb5f26057b5a admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:13:58.813059 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 781/1573] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:13:58 2022] POST /identity/v3/users => generated 299 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:13:58.813059 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c87d76e-336c-457b-a110-734c65be71fc admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:13:58.821453 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c87d76e-336c-457b-a110-734c65be71fc admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:13:58.830577 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:13:58.838775 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:13:58.861674 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16d0e56-0061-48d3-bbce-bb5f26057b5a admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:13:58.866266 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e0239-f189-4fbd-bb29-3086d3c6e387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:13:58.866873 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e0239-f189-4fbd-bb29-3086d3c6e387 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:58.867148 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e0239-f189-4fbd-bb29-3086d3c6e387 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:58.867444 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e0239-f189-4fbd-bb29-3086d3c6e387 admin admin] PATH_INFO: `/v3/projects` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:58.868324 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227e0239-f189-4fbd-bb29-3086d3c6e387 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:13:58.870481 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227e0239-f189-4fbd-bb29-3086d3c6e387 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:13:58.883993 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f83013-7d5e-4d8c-a75e-6ff6189b5d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:13:58.885145 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f83013-7d5e-4d8c-a75e-6ff6189b5d52 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:58.885424 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f83013-7d5e-4d8c-a75e-6ff6189b5d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:58.885726 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f83013-7d5e-4d8c-a75e-6ff6189b5d52 admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:58.886477 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f83013-7d5e-4d8c-a75e-6ff6189b5d52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:13:58.888846 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f83013-7d5e-4d8c-a75e-6ff6189b5d52 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:13:58.897919 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 793/1574] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:13:58 2022] POST /identity/v3/projects => generated 375 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:13:58.900663 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e0239-f189-4fbd-bb29-3086d3c6e387 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:13:58.921204 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 782/1575] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:13:58 2022] POST /identity/v3/projects => generated 383 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:13:58.921204 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f83013-7d5e-4d8c-a75e-6ff6189b5d52 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:13:58.970179 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e0239-f189-4fbd-bb29-3086d3c6e387 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:13:58.980481 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f83013-7d5e-4d8c-a75e-6ff6189b5d52 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:13:58.985071 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217fc1d4-d919-495a-a998-3e158f67e3ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:13:58.985671 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217fc1d4-d919-495a-a998-3e158f67e3ad admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:58.985954 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217fc1d4-d919-495a-a998-3e158f67e3ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:58.986225 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217fc1d4-d919-495a-a998-3e158f67e3ad admin admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:58.986868 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217fc1d4-d919-495a-a998-3e158f67e3ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:13:58.988821 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217fc1d4-d919-495a-a998-3e158f67e3ad admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:13:59.011252 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 783/1576] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:13:58 2022] POST /identity/v3/users => generated 302 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:13:59.012465 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217fc1d4-d919-495a-a998-3e158f67e3ad admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:13:59.021762 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217fc1d4-d919-495a-a998-3e158f67e3ad admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:13:59.027339 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499972b5-f248-44dd-9257-9658e4427737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:13:59.027924 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499972b5-f248-44dd-9257-9658e4427737 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:59.028208 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499972b5-f248-44dd-9257-9658e4427737 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:59.028482 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499972b5-f248-44dd-9257-9658e4427737 admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:59.029094 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499972b5-f248-44dd-9257-9658e4427737 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:13:59.029701 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39487568-b4c2-4e9e-b370-eb9aa3dad7d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:13:59.030307 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39487568-b4c2-4e9e-b370-eb9aa3dad7d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:59.030588 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39487568-b4c2-4e9e-b370-eb9aa3dad7d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:59.030860 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39487568-b4c2-4e9e-b370-eb9aa3dad7d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:59.030972 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499972b5-f248-44dd-9257-9658e4427737 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:13:59.031524 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39487568-b4c2-4e9e-b370-eb9aa3dad7d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:13:59.033417 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39487568-b4c2-4e9e-b370-eb9aa3dad7d3 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:13:59.058445 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 784/1577] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:13:59 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:13:59.059728 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 794/1578] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:13:58 2022] POST /identity/v3/projects => generated 377 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:13:59.060158 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499972b5-f248-44dd-9257-9658e4427737 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:13:59.063639 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39487568-b4c2-4e9e-b370-eb9aa3dad7d3 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:13:59.116902 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:13:59.124640 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499972b5-f248-44dd-9257-9658e4427737 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:13:59.129906 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4032b27c-221e-4a1e-8377-4a24b8268568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:13:59.130081 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39487568-b4c2-4e9e-b370-eb9aa3dad7d3 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:13:59.130522 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4032b27c-221e-4a1e-8377-4a24b8268568 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:59.130805 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4032b27c-221e-4a1e-8377-4a24b8268568 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:59.131079 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4032b27c-221e-4a1e-8377-4a24b8268568 admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:59.131732 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4032b27c-221e-4a1e-8377-4a24b8268568 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:13:59.133646 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4032b27c-221e-4a1e-8377-4a24b8268568 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:13:59.134881 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bd44a7-bbbc-4073-9f3b-35908c0a9a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:13:59.135619 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd44a7-bbbc-4073-9f3b-35908c0a9a61 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:59.135900 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd44a7-bbbc-4073-9f3b-35908c0a9a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:59.136172 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd44a7-bbbc-4073-9f3b-35908c0a9a61 admin admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:59.136784 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bd44a7-bbbc-4073-9f3b-35908c0a9a61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:13:59.139245 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bd44a7-bbbc-4073-9f3b-35908c0a9a61 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:13:59.148495 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:13:59.161698 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 785/1579] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:13:59 2022] POST /identity/v3/projects => generated 365 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:13:59.166251 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 795/1580] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:13:59 2022] POST /identity/v3/users => generated 296 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:13:59.166423 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4032b27c-221e-4a1e-8377-4a24b8268568 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:13:59.166538 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bd44a7-bbbc-4073-9f3b-35908c0a9a61 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:13:59.175143 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bd44a7-bbbc-4073-9f3b-35908c0a9a61 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:13:59.226496 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4032b27c-221e-4a1e-8377-4a24b8268568 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:13:59.232040 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4f159f-5a62-4eae-828c-5ce1476dd526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:13:59.232664 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4f159f-5a62-4eae-828c-5ce1476dd526 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:59.232938 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4f159f-5a62-4eae-828c-5ce1476dd526 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:59.233207 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4f159f-5a62-4eae-828c-5ce1476dd526 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:59.233840 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4f159f-5a62-4eae-828c-5ce1476dd526 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:13:59.234223 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e42dd9-e7fb-4c0d-b7d0-093588848ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:13:59.234839 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e42dd9-e7fb-4c0d-b7d0-093588848ff1 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:59.235103 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e42dd9-e7fb-4c0d-b7d0-093588848ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:59.235406 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e42dd9-e7fb-4c0d-b7d0-093588848ff1 admin admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:59.235531 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4f159f-5a62-4eae-828c-5ce1476dd526 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:13:59.236028 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e42dd9-e7fb-4c0d-b7d0-093588848ff1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:13:59.237930 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e42dd9-e7fb-4c0d-b7d0-093588848ff1 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:13:59.245478 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 786/1581] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:13:59.248188 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4f159f-5a62-4eae-828c-5ce1476dd526 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:13:59.258398 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4f159f-5a62-4eae-828c-5ce1476dd526 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:13:59.261070 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 796/1582] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:13:59 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:13:59.262004 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e42dd9-e7fb-4c0d-b7d0-093588848ff1 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:13:59.263415 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9bbc1e-ab83-4841-86c9-47f838bc2fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:13:59.264009 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9bbc1e-ab83-4841-86c9-47f838bc2fb6 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:59.264284 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9bbc1e-ab83-4841-86c9-47f838bc2fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:59.264562 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9bbc1e-ab83-4841-86c9-47f838bc2fb6 admin admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:59.265173 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9bbc1e-ab83-4841-86c9-47f838bc2fb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:13:59.267094 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9bbc1e-ab83-4841-86c9-47f838bc2fb6 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:13:59.271702 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e42dd9-e7fb-4c0d-b7d0-093588848ff1 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:13:59.276919 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dd6dc1-3d87-4702-8ae1-4d8fe2963ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:13:59.277502 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd6dc1-3d87-4702-8ae1-4d8fe2963ceb admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:59.277777 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd6dc1-3d87-4702-8ae1-4d8fe2963ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:59.278077 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd6dc1-3d87-4702-8ae1-4d8fe2963ceb admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:59.278798 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dd6dc1-3d87-4702-8ae1-4d8fe2963ceb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:13:59.280643 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dd6dc1-3d87-4702-8ae1-4d8fe2963ceb admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:13:59.292841 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 787/1583] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:13:59 2022] POST /identity/v3/users => generated 292 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:13:59.292841 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9bbc1e-ab83-4841-86c9-47f838bc2fb6 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:13:59.293080 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 797/1584] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:13:59.293593 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dd6dc1-3d87-4702-8ae1-4d8fe2963ceb admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:13:59.301229 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9bbc1e-ab83-4841-86c9-47f838bc2fb6 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:13:59.304256 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dd6dc1-3d87-4702-8ae1-4d8fe2963ceb admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:13:59.306144 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b148b-83f0-477f-b61a-25e495d7def1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:13:59.306731 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b148b-83f0-477f-b61a-25e495d7def1 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:59.307007 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b148b-83f0-477f-b61a-25e495d7def1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:59.307312 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b148b-83f0-477f-b61a-25e495d7def1 admin admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:59.307920 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042b148b-83f0-477f-b61a-25e495d7def1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:13:59.309268 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b2fc0b-2a38-42a3-8a60-33aeefc96b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:13:59.309805 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042b148b-83f0-477f-b61a-25e495d7def1 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:13:59.309917 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b2fc0b-2a38-42a3-8a60-33aeefc96b97 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:59.310112 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b2fc0b-2a38-42a3-8a60-33aeefc96b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:59.310385 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b2fc0b-2a38-42a3-8a60-33aeefc96b97 admin admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:59.310997 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b2fc0b-2a38-42a3-8a60-33aeefc96b97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:13:59.312910 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b2fc0b-2a38-42a3-8a60-33aeefc96b97 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:13:59.332343 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 788/1585] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:13:59 2022] POST /identity/v3/users => generated 290 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:13:59.333094 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b148b-83f0-477f-b61a-25e495d7def1 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:13:59.338374 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 798/1586] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:13:59 2022] POST /identity/v3/users => generated 289 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:13:59.338374 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b2fc0b-2a38-42a3-8a60-33aeefc96b97 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:13:59.343491 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b148b-83f0-477f-b61a-25e495d7def1 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:13:59.346383 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b2fc0b-2a38-42a3-8a60-33aeefc96b97 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:13:59.348084 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b16a137-98ae-4305-a876-1c7998b937b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:13:59.348680 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16a137-98ae-4305-a876-1c7998b937b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:59.348957 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16a137-98ae-4305-a876-1c7998b937b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:59.349227 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16a137-98ae-4305-a876-1c7998b937b3 admin admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:59.349840 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b16a137-98ae-4305-a876-1c7998b937b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:13:59.350983 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13919be-0256-498b-8424-fb3d01cf9125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:13:59.351591 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13919be-0256-498b-8424-fb3d01cf9125 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:59.351792 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b16a137-98ae-4305-a876-1c7998b937b3 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:13:59.351902 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13919be-0256-498b-8424-fb3d01cf9125 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:59.352123 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13919be-0256-498b-8424-fb3d01cf9125 admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:59.352791 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13919be-0256-498b-8424-fb3d01cf9125 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:13:59.354878 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13919be-0256-498b-8424-fb3d01cf9125 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:13:59.365285 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 799/1587] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:13:59.366160 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13919be-0256-498b-8424-fb3d01cf9125 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:13:59.375735 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 789/1588] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:13:59 2022] POST /identity/v3/users => generated 283 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:13:59.379013 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13919be-0256-498b-8424-fb3d01cf9125 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:13:59.379460 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b16a137-98ae-4305-a876-1c7998b937b3 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:13:59.387654 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038ad2d3-ef7b-45ff-988e-eee0825bc2a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:13:59.388344 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ad2d3-ef7b-45ff-988e-eee0825bc2a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:59.388685 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ad2d3-ef7b-45ff-988e-eee0825bc2a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:59.388900 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ad2d3-ef7b-45ff-988e-eee0825bc2a7 admin admin] PATH_INFO: `/v3/projects/a0e89065424a4994842f3469465e394c/users/1c887b76db5f48b98a5dbe1632e7eac0/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:59.389693 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b16a137-98ae-4305-a876-1c7998b937b3 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:13:59.394307 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4880df7c-159f-47b5-ab12-dbba2afe9666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:13:59.394913 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4880df7c-159f-47b5-ab12-dbba2afe9666 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:59.395194 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4880df7c-159f-47b5-ab12-dbba2afe9666 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:59.395490 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4880df7c-159f-47b5-ab12-dbba2afe9666 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:59.396126 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4880df7c-159f-47b5-ab12-dbba2afe9666 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:13:59.397716 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4880df7c-159f-47b5-ab12-dbba2afe9666 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:13:59.404795 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038ad2d3-ef7b-45ff-988e-eee0825bc2a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0e89065424a4994842f3469465e394c, user_id=1c887b76db5f48b98a5dbe1632e7eac0, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:13:59.405814 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: WARNING py.warnings [None req-038ad2d3-ef7b-45ff-988e-eee0825bc2a7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 12:13:59.405814 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: warnings.warn(msg) Aug 26 12:13:59.405814 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: Aug 26 12:13:59.407803 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038ad2d3-ef7b-45ff-988e-eee0825bc2a7 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:13:59.408423 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 790/1589] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:13:59.409356 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4880df7c-159f-47b5-ab12-dbba2afe9666 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:13:59.420008 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4880df7c-159f-47b5-ab12-dbba2afe9666 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:13:59.423384 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 800/1590] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:13:59 2022] PUT /identity/v3/projects/a0e89065424a4994842f3469465e394c/users/1c887b76db5f48b98a5dbe1632e7eac0/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:13:59.424281 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038ad2d3-ef7b-45ff-988e-eee0825bc2a7 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:13:59.479337 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a1fe4d-c03b-4b1e-a3d8-ea086082b468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:13:59.479944 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a1fe4d-c03b-4b1e-a3d8-ea086082b468 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:59.480223 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a1fe4d-c03b-4b1e-a3d8-ea086082b468 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:59.480497 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a1fe4d-c03b-4b1e-a3d8-ea086082b468 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:59.481127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a1fe4d-c03b-4b1e-a3d8-ea086082b468 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:13:59.482714 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a1fe4d-c03b-4b1e-a3d8-ea086082b468 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:13:59.487440 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038ad2d3-ef7b-45ff-988e-eee0825bc2a7 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:13:59.492296 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 791/1591] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:13:59.492646 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef0d76-d8bd-4e44-85e3-24210956a20c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:13:59.493362 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a1fe4d-c03b-4b1e-a3d8-ea086082b468 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:13:59.494520 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef0d76-d8bd-4e44-85e3-24210956a20c admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:59.494793 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef0d76-d8bd-4e44-85e3-24210956a20c admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:59.495106 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef0d76-d8bd-4e44-85e3-24210956a20c admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:59.495868 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ef0d76-d8bd-4e44-85e3-24210956a20c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:13:59.497581 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ef0d76-d8bd-4e44-85e3-24210956a20c admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:13:59.504524 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a1fe4d-c03b-4b1e-a3d8-ea086082b468 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:13:59.507926 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 801/1592] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:13:59.508794 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef0d76-d8bd-4e44-85e3-24210956a20c admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:13:59.512606 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8729e07a-d16d-476a-aa5d-46c7ce095086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:13:59.513233 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8729e07a-d16d-476a-aa5d-46c7ce095086 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:59.513513 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8729e07a-d16d-476a-aa5d-46c7ce095086 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:59.513783 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8729e07a-d16d-476a-aa5d-46c7ce095086 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:59.514470 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8729e07a-d16d-476a-aa5d-46c7ce095086 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:13:59.514710 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:13:59.516345 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8729e07a-d16d-476a-aa5d-46c7ce095086 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:13:59.521190 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef0d76-d8bd-4e44-85e3-24210956a20c admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:13:59.526159 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30938b54-31aa-44af-928b-99a7d292eef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:13:59.526737 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30938b54-31aa-44af-928b-99a7d292eef2 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:59.527020 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30938b54-31aa-44af-928b-99a7d292eef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:59.527384 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30938b54-31aa-44af-928b-99a7d292eef2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:59.528334 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30938b54-31aa-44af-928b-99a7d292eef2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:13:59.528907 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 792/1593] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:13:59.529805 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8729e07a-d16d-476a-aa5d-46c7ce095086 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:13:59.532991 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30938b54-31aa-44af-928b-99a7d292eef2 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:13:59.539646 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8729e07a-d16d-476a-aa5d-46c7ce095086 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:13:59.543574 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 802/1594] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:13:59.544835 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30938b54-31aa-44af-928b-99a7d292eef2 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:13:59.545520 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9110fcb1-abbf-4c9c-ae10-c1d73c4e3158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:13:59.546369 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110fcb1-abbf-4c9c-ae10-c1d73c4e3158 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:59.546866 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110fcb1-abbf-4c9c-ae10-c1d73c4e3158 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:59.547354 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110fcb1-abbf-4c9c-ae10-c1d73c4e3158 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:59.548049 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9110fcb1-abbf-4c9c-ae10-c1d73c4e3158 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:13:59.549585 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9110fcb1-abbf-4c9c-ae10-c1d73c4e3158 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:13:59.559245 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30938b54-31aa-44af-928b-99a7d292eef2 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:13:59.565351 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 793/1595] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:13:59.565351 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9110fcb1-abbf-4c9c-ae10-c1d73c4e3158 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:13:59.565591 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0305d403-ba62-445a-bbb2-848fd22300cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:13:59.565591 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0305d403-ba62-445a-bbb2-848fd22300cb admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:59.565591 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0305d403-ba62-445a-bbb2-848fd22300cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:59.565591 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0305d403-ba62-445a-bbb2-848fd22300cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:59.565959 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0305d403-ba62-445a-bbb2-848fd22300cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:13:59.567249 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0305d403-ba62-445a-bbb2-848fd22300cb admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:13:59.568194 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:13:59.574432 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9110fcb1-abbf-4c9c-ae10-c1d73c4e3158 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:13:59.579051 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 803/1596] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:13:59.579972 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0305d403-ba62-445a-bbb2-848fd22300cb admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:13:59.583698 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58857e66-0c49-427f-a713-6d5a394c66f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:13:59.584304 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58857e66-0c49-427f-a713-6d5a394c66f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:59.584586 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58857e66-0c49-427f-a713-6d5a394c66f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:59.584859 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58857e66-0c49-427f-a713-6d5a394c66f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:59.585504 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58857e66-0c49-427f-a713-6d5a394c66f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:13:59.587120 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58857e66-0c49-427f-a713-6d5a394c66f8 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:13:59.589629 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0305d403-ba62-445a-bbb2-848fd22300cb admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:13:59.594181 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cbd9f8-6a87-46eb-b066-8cfc49b7f2ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:13:59.594758 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cbd9f8-6a87-46eb-b066-8cfc49b7f2ef admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:59.595032 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cbd9f8-6a87-46eb-b066-8cfc49b7f2ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:59.595334 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cbd9f8-6a87-46eb-b066-8cfc49b7f2ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:59.595958 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cbd9f8-6a87-46eb-b066-8cfc49b7f2ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:13:59.597561 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cbd9f8-6a87-46eb-b066-8cfc49b7f2ef admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:13:59.597970 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 794/1597] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:13:59.598671 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58857e66-0c49-427f-a713-6d5a394c66f8 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:13:59.611796 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 804/1598] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:13:59.611796 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cbd9f8-6a87-46eb-b066-8cfc49b7f2ef admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:13:59.612100 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58857e66-0c49-427f-a713-6d5a394c66f8 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:13:59.616435 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435e0586-0802-49ab-9d7f-06646200d57b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:13:59.617030 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e0586-0802-49ab-9d7f-06646200d57b admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:59.617306 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e0586-0802-49ab-9d7f-06646200d57b admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:59.617576 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e0586-0802-49ab-9d7f-06646200d57b admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:59.619099 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435e0586-0802-49ab-9d7f-06646200d57b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:13:59.620776 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435e0586-0802-49ab-9d7f-06646200d57b admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:13:59.620975 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cbd9f8-6a87-46eb-b066-8cfc49b7f2ef admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:13:59.625603 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba8121f-cc95-4dc7-901b-e47b13def101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:13:59.626324 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba8121f-cc95-4dc7-901b-e47b13def101 admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:59.626523 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba8121f-cc95-4dc7-901b-e47b13def101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:59.626790 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba8121f-cc95-4dc7-901b-e47b13def101 admin admin] PATH_INFO: `/v3/projects/5a81358b458943c3abac971c3dcf9636/users/be2144973da4484b934173964168da80/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:59.631577 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 795/1599] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:13:59.635864 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435e0586-0802-49ab-9d7f-06646200d57b admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:13:59.642501 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba8121f-cc95-4dc7-901b-e47b13def101 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a81358b458943c3abac971c3dcf9636, user_id=be2144973da4484b934173964168da80, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:13:59.645151 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435e0586-0802-49ab-9d7f-06646200d57b admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:13:59.645295 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba8121f-cc95-4dc7-901b-e47b13def101 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:13:59.650251 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69346b98-8afd-41a3-a9cc-75d6d7d5e2e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:13:59.650942 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69346b98-8afd-41a3-a9cc-75d6d7d5e2e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:59.651264 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69346b98-8afd-41a3-a9cc-75d6d7d5e2e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:59.651536 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69346b98-8afd-41a3-a9cc-75d6d7d5e2e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:59.652260 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69346b98-8afd-41a3-a9cc-75d6d7d5e2e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:13:59.653982 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69346b98-8afd-41a3-a9cc-75d6d7d5e2e3 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:13:59.659506 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 805/1600] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:13:59 2022] PUT /identity/v3/projects/5a81358b458943c3abac971c3dcf9636/users/be2144973da4484b934173964168da80/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:13:59.660377 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba8121f-cc95-4dc7-901b-e47b13def101 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:13:59.668848 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 796/1601] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:13:59.669676 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69346b98-8afd-41a3-a9cc-75d6d7d5e2e3 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:13:59.679412 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69346b98-8afd-41a3-a9cc-75d6d7d5e2e3 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:13:59.726545 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba8121f-cc95-4dc7-901b-e47b13def101 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:13:59.732952 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b591273-3be3-4ce0-ad3d-5ad9781d99fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:13:59.733555 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b591273-3be3-4ce0-ad3d-5ad9781d99fe admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:59.733834 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b591273-3be3-4ce0-ad3d-5ad9781d99fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:59.734112 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b591273-3be3-4ce0-ad3d-5ad9781d99fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:59.734701 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc55034c-6fb2-4d0a-847f-e35249e76ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:13:59.735382 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc55034c-6fb2-4d0a-847f-e35249e76ab3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:59.735594 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b591273-3be3-4ce0-ad3d-5ad9781d99fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:13:59.735710 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc55034c-6fb2-4d0a-847f-e35249e76ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:59.735919 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc55034c-6fb2-4d0a-847f-e35249e76ab3 admin admin] PATH_INFO: `/v3/projects/984b6ea268f04e8989283e6397b68370/users/a02508c20bd547708638c1875101da6f/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:59.737188 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b591273-3be3-4ce0-ad3d-5ad9781d99fe admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:13:59.750822 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 806/1602] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:13:59.751620 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc55034c-6fb2-4d0a-847f-e35249e76ab3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=984b6ea268f04e8989283e6397b68370, user_id=a02508c20bd547708638c1875101da6f, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:13:59.751770 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b591273-3be3-4ce0-ad3d-5ad9781d99fe admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:13:59.754389 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc55034c-6fb2-4d0a-847f-e35249e76ab3 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:13:59.762368 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b591273-3be3-4ce0-ad3d-5ad9781d99fe admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:13:59.768796 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 797/1603] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:13:59 2022] PUT /identity/v3/projects/984b6ea268f04e8989283e6397b68370/users/a02508c20bd547708638c1875101da6f/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:13:59.769585 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc55034c-6fb2-4d0a-847f-e35249e76ab3 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:13:59.822550 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2509fed4-1010-4ea8-8e72-c90f94916343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:13:59.823147 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2509fed4-1010-4ea8-8e72-c90f94916343 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:59.823460 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2509fed4-1010-4ea8-8e72-c90f94916343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:59.823740 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2509fed4-1010-4ea8-8e72-c90f94916343 admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:59.824376 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2509fed4-1010-4ea8-8e72-c90f94916343 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:13:59.825971 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2509fed4-1010-4ea8-8e72-c90f94916343 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:13:59.832773 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc55034c-6fb2-4d0a-847f-e35249e76ab3 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:13:59.835670 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 807/1604] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:13:59.836514 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2509fed4-1010-4ea8-8e72-c90f94916343 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:13:59.837921 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe9f515-1b06-4ed3-b274-1c54fe209f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:13:59.838583 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe9f515-1b06-4ed3-b274-1c54fe209f35 admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:59.838860 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe9f515-1b06-4ed3-b274-1c54fe209f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:59.839133 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe9f515-1b06-4ed3-b274-1c54fe209f35 admin admin] PATH_INFO: `/v3/projects/aff0bc2a1f414ae2ad59f54e2bb5ff36/users/cee554500b02434c8c5aad6d498bc287/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:59.846922 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2509fed4-1010-4ea8-8e72-c90f94916343 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:13:59.852133 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f394281-a000-4685-892b-fb4d848f7691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:13:59.852828 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f394281-a000-4685-892b-fb4d848f7691 admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:59.853107 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f394281-a000-4685-892b-fb4d848f7691 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:59.853384 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f394281-a000-4685-892b-fb4d848f7691 admin admin] PATH_INFO: `/v3/projects/a0e89065424a4994842f3469465e394c/users/1c887b76db5f48b98a5dbe1632e7eac0/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:59.854694 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe9f515-1b06-4ed3-b274-1c54fe209f35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aff0bc2a1f414ae2ad59f54e2bb5ff36, user_id=cee554500b02434c8c5aad6d498bc287, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:13:59.855319 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f394281-a000-4685-892b-fb4d848f7691 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0e89065424a4994842f3469465e394c, user_id=1c887b76db5f48b98a5dbe1632e7eac0, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:13:59.857596 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe9f515-1b06-4ed3-b274-1c54fe209f35 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:13:59.857913 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f394281-a000-4685-892b-fb4d848f7691 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:13:59.871226 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 808/1605] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:13:59 2022] PUT /identity/v3/projects/a0e89065424a4994842f3469465e394c/users/1c887b76db5f48b98a5dbe1632e7eac0/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:13:59.872238 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 798/1606] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:13:59 2022] PUT /identity/v3/projects/aff0bc2a1f414ae2ad59f54e2bb5ff36/users/cee554500b02434c8c5aad6d498bc287/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:13:59.872379 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f394281-a000-4685-892b-fb4d848f7691 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:13:59.873246 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe76df7-552d-4f7b-92a0-b639470e3c55 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:13:59.873532 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe76df7-552d-4f7b-92a0-b639470e3c55 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:13:59.873807 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe76df7-552d-4f7b-92a0-b639470e3c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:13:59.907079 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-5fe76df7-552d-4f7b-92a0-b639470e3c55 None None] MFA Rules not processed for user `be2144973da4484b934173964168da80`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:13:59.908237 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-5fe76df7-552d-4f7b-92a0-b639470e3c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:13:59.909788 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:13:59.932721 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:13:59.948878 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f394281-a000-4685-892b-fb4d848f7691 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:13:59.973887 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 799/1607] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:13:59 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:13:59.974700 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe76df7-552d-4f7b-92a0-b639470e3c55 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:00.008573 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139e1ea7-a179-4943-bf36-24c9974f7a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:00.009776 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139e1ea7-a179-4943-bf36-24c9974f7a1c admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:00.010050 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139e1ea7-a179-4943-bf36-24c9974f7a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:00.010316 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139e1ea7-a179-4943-bf36-24c9974f7a1c admin admin] PATH_INFO: `/v3/projects/9574496fe7ba4144a05fca690cc3486f/users/36a27ca98a2e4521b269be48c9ec4cb0/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:00.024675 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139e1ea7-a179-4943-bf36-24c9974f7a1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9574496fe7ba4144a05fca690cc3486f, user_id=36a27ca98a2e4521b269be48c9ec4cb0, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:00.027853 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139e1ea7-a179-4943-bf36-24c9974f7a1c admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:00.037155 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe76df7-552d-4f7b-92a0-b639470e3c55 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:00.041377 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 809/1608] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:13:59 2022] PUT /identity/v3/projects/9574496fe7ba4144a05fca690cc3486f/users/36a27ca98a2e4521b269be48c9ec4cb0/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:00.041927 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139e1ea7-a179-4943-bf36-24c9974f7a1c admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:00.094937 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4a84b1-0962-4038-97c1-9da991dc70a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:00.095723 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4a84b1-0962-4038-97c1-9da991dc70a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:00.096063 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4a84b1-0962-4038-97c1-9da991dc70a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:00.096385 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4a84b1-0962-4038-97c1-9da991dc70a8 admin admin] PATH_INFO: `/v3/projects/e58ce6250ef341b6bdc369d9475f0670/users/ff96b716abcd4565866fc3c24cbe91e5/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:00.106564 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139e1ea7-a179-4943-bf36-24c9974f7a1c admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:00.110659 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4a84b1-0962-4038-97c1-9da991dc70a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e58ce6250ef341b6bdc369d9475f0670, user_id=ff96b716abcd4565866fc3c24cbe91e5, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:00.111138 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06f1191-c989-4892-92c5-500060071a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:00.111857 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f1191-c989-4892-92c5-500060071a66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:00.112136 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f1191-c989-4892-92c5-500060071a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:00.112406 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f1191-c989-4892-92c5-500060071a66 admin admin] PATH_INFO: `/v3/projects/99e42555cbcd4716a1be1075f4a2b3b1/users/2d9f1a5e4e77467f8a691b634a841de2/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:00.113391 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4a84b1-0962-4038-97c1-9da991dc70a8 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:00.127868 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 800/1609] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:13:59 2022] PUT /identity/v3/projects/e58ce6250ef341b6bdc369d9475f0670/users/ff96b716abcd4565866fc3c24cbe91e5/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:00.128086 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06f1191-c989-4892-92c5-500060071a66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99e42555cbcd4716a1be1075f4a2b3b1, user_id=2d9f1a5e4e77467f8a691b634a841de2, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:00.129180 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c2ffa4-2fd0-4524-ba45-0ff43411e446 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:00.129467 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c2ffa4-2fd0-4524-ba45-0ff43411e446 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:00.129739 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c2ffa4-2fd0-4524-ba45-0ff43411e446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:00.130719 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06f1191-c989-4892-92c5-500060071a66 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:00.145469 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 810/1610] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:14:00 2022] PUT /identity/v3/projects/99e42555cbcd4716a1be1075f4a2b3b1/users/2d9f1a5e4e77467f8a691b634a841de2/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:00.146385 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06f1191-c989-4892-92c5-500060071a66 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:00.152970 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-52c2ffa4-2fd0-4524-ba45-0ff43411e446 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:00.154175 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-52c2ffa4-2fd0-4524-ba45-0ff43411e446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:00.187089 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:00.204291 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:00.211910 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06f1191-c989-4892-92c5-500060071a66 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:00.216717 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d41cf0-ba1f-48d8-b11c-da2d0ee81de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:00.217085 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 801/1611] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:00.217372 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d41cf0-ba1f-48d8-b11c-da2d0ee81de5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:00.217652 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d41cf0-ba1f-48d8-b11c-da2d0ee81de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:00.217930 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d41cf0-ba1f-48d8-b11c-da2d0ee81de5 admin admin] PATH_INFO: `/v3/projects/d112018ddd254158ac5d36977c90374e/users/dca46c79bb614edb9c544031bbfad47e/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:00.218289 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d34f55-c945-40cb-835a-f7494c68d9e7 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:00.218570 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d34f55-c945-40cb-835a-f7494c68d9e7 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:00.218840 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d34f55-c945-40cb-835a-f7494c68d9e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:00.234729 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d41cf0-ba1f-48d8-b11c-da2d0ee81de5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d112018ddd254158ac5d36977c90374e, user_id=dca46c79bb614edb9c544031bbfad47e, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:00.237557 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d41cf0-ba1f-48d8-b11c-da2d0ee81de5 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:00.247900 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-14d34f55-c945-40cb-835a-f7494c68d9e7 None None] MFA Rules not processed for user `1c887b76db5f48b98a5dbe1632e7eac0`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:00.249120 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-14d34f55-c945-40cb-835a-f7494c68d9e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:00.251481 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 811/1612] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:14:00 2022] PUT /identity/v3/projects/d112018ddd254158ac5d36977c90374e/users/dca46c79bb614edb9c544031bbfad47e/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:00.252519 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b12552-6a1a-4dee-af49-b87a4314aaa2 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:00.252923 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b12552-6a1a-4dee-af49-b87a4314aaa2 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:00.253200 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b12552-6a1a-4dee-af49-b87a4314aaa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:00.284333 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-b5b12552-6a1a-4dee-af49-b87a4314aaa2 None None] MFA Rules not processed for user `cee554500b02434c8c5aad6d498bc287`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:00.285595 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-b5b12552-6a1a-4dee-af49-b87a4314aaa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:00.325411 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 802/1613] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:00 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:00.326228 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d34f55-c945-40cb-835a-f7494c68d9e7 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:00.357009 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 812/1614] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:00.358740 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d6175-a55c-4f32-a1ef-6c3cb9539f22 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:00.359024 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d6175-a55c-4f32-a1ef-6c3cb9539f22 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:00.359375 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d6175-a55c-4f32-a1ef-6c3cb9539f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:00.379043 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-5e3d6175-a55c-4f32-a1ef-6c3cb9539f22 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:00.380249 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-5e3d6175-a55c-4f32-a1ef-6c3cb9539f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:00.404075 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d34f55-c945-40cb-835a-f7494c68d9e7 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:00.408664 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f5bdb0-2d48-4a88-b904-639b82cd0a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:00.409288 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f5bdb0-2d48-4a88-b904-639b82cd0a43 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:00.409569 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f5bdb0-2d48-4a88-b904-639b82cd0a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:00.409835 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f5bdb0-2d48-4a88-b904-639b82cd0a43 admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:00.410454 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f5bdb0-2d48-4a88-b904-639b82cd0a43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:00.412397 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f5bdb0-2d48-4a88-b904-639b82cd0a43 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:00.441231 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 813/1615] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:00.444667 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48feb7f9-3937-4a4b-8ba6-133d38366c68 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:00.444667 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48feb7f9-3937-4a4b-8ba6-133d38366c68 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:00.444848 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 803/1616] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:14:00 2022] POST /identity/v3/projects => generated 403 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:14:00.445025 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48feb7f9-3937-4a4b-8ba6-133d38366c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:00.446248 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f5259-18bb-4ffc-8f70-d3d3b9e59b75 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:00.446597 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f5259-18bb-4ffc-8f70-d3d3b9e59b75 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:00.447070 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f5259-18bb-4ffc-8f70-d3d3b9e59b75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:00.464154 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-48feb7f9-3937-4a4b-8ba6-133d38366c68 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:00.465070 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-4d9f5259-18bb-4ffc-8f70-d3d3b9e59b75 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:00.465337 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-48feb7f9-3937-4a4b-8ba6-133d38366c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:00.466262 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-4d9f5259-18bb-4ffc-8f70-d3d3b9e59b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:00.510306 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:00.525320 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 814/1617] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:00.526128 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48feb7f9-3937-4a4b-8ba6-133d38366c68 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:00.527405 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-48feb7f9-3937-4a4b-8ba6-133d38366c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:00.530083 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 804/1618] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:00.534643 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f05989-a5ae-4e1b-8663-93701eef0865 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:00.534988 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f05989-a5ae-4e1b-8663-93701eef0865 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:00.535296 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f05989-a5ae-4e1b-8663-93701eef0865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:00.549806 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:00.565058 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-e5f05989-a5ae-4e1b-8663-93701eef0865 None None] MFA Rules not processed for user `dca46c79bb614edb9c544031bbfad47e`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:00.566262 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-e5f05989-a5ae-4e1b-8663-93701eef0865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:00.583964 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48feb7f9-3937-4a4b-8ba6-133d38366c68 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:00.588151 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a917ee-58ab-4fa6-b023-1b6bfd69bb89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:00.588746 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a917ee-58ab-4fa6-b023-1b6bfd69bb89 service neutron] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:00.589019 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a917ee-58ab-4fa6-b023-1b6bfd69bb89 service neutron] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:00.589293 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a917ee-58ab-4fa6-b023-1b6bfd69bb89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:00.598200 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a917ee-58ab-4fa6-b023-1b6bfd69bb89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:00.599705 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a917ee-58ab-4fa6-b023-1b6bfd69bb89 service neutron] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:00.609716 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 815/1619] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:14:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:14:00.610593 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a917ee-58ab-4fa6-b023-1b6bfd69bb89 service neutron] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:00.632709 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 805/1620] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:00 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:00.634516 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60fbe0d-a2f7-4e21-bcc7-7e3c4f07e428 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:00.634774 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60fbe0d-a2f7-4e21-bcc7-7e3c4f07e428 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:00.635043 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60fbe0d-a2f7-4e21-bcc7-7e3c4f07e428 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:00.636505 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 806/1621] 10.222.0.61 () {54 vars in 962 bytes} [Fri Aug 26 12:14:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:14:00.637922 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60fbe0d-a2f7-4e21-bcc7-7e3c4f07e428 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:00.653169 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60fbe0d-a2f7-4e21-bcc7-7e3c4f07e428 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:00.657864 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9418ffb4-b728-453b-9a9c-41aa797714ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:00.658472 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418ffb4-b728-453b-9a9c-41aa797714ac admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:00.658753 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418ffb4-b728-453b-9a9c-41aa797714ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:00.659170 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418ffb4-b728-453b-9a9c-41aa797714ac admin admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:00.660438 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9418ffb4-b728-453b-9a9c-41aa797714ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:00.661779 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9418ffb4-b728-453b-9a9c-41aa797714ac admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:00.687961 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a917ee-58ab-4fa6-b023-1b6bfd69bb89 service neutron] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:00.691605 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 807/1622] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:14:00 2022] POST /identity/v3/users => generated 308 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:14:00.691605 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9418ffb4-b728-453b-9a9c-41aa797714ac admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:00.694687 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b346f47-7cd3-4046-a3c1-33b0e2cb75d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:00.695425 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b346f47-7cd3-4046-a3c1-33b0e2cb75d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:00.695780 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b346f47-7cd3-4046-a3c1-33b0e2cb75d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:00.696100 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b346f47-7cd3-4046-a3c1-33b0e2cb75d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:00.703004 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9418ffb4-b728-453b-9a9c-41aa797714ac admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:00.707678 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60233e4-cf2d-4e73-a599-2baa2f5b3725 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:00.708309 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60233e4-cf2d-4e73-a599-2baa2f5b3725 service neutron] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:00.708662 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60233e4-cf2d-4e73-a599-2baa2f5b3725 service neutron] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:00.708918 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60233e4-cf2d-4e73-a599-2baa2f5b3725 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:00.720958 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60233e4-cf2d-4e73-a599-2baa2f5b3725 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:00.723118 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60233e4-cf2d-4e73-a599-2baa2f5b3725 service neutron] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:00.733166 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 808/1623] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:14:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:14:00.733919 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60233e4-cf2d-4e73-a599-2baa2f5b3725 service neutron] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:00.752265 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60233e4-cf2d-4e73-a599-2baa2f5b3725 service neutron] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:00.756641 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1833ed-a2cd-41a9-9e17-a53d2f46c812 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:00.757492 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1833ed-a2cd-41a9-9e17-a53d2f46c812 service neutron] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:00.757778 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1833ed-a2cd-41a9-9e17-a53d2f46c812 service neutron] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:00.758052 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1833ed-a2cd-41a9-9e17-a53d2f46c812 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:00.770233 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1833ed-a2cd-41a9-9e17-a53d2f46c812 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:00.772090 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1833ed-a2cd-41a9-9e17-a53d2f46c812 service neutron] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:00.779334 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b346f47-7cd3-4046-a3c1-33b0e2cb75d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:00.780790 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b346f47-7cd3-4046-a3c1-33b0e2cb75d8 service cinder] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:00.783735 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 809/1624] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:14:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:14:00.784414 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1833ed-a2cd-41a9-9e17-a53d2f46c812 service neutron] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:00.794203 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 816/1625] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:14:00 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:14:00.794755 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1833ed-a2cd-41a9-9e17-a53d2f46c812 service neutron] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:00.798898 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f395cc-4add-489d-b4a9-f116c0b0d388 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:00.799546 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f395cc-4add-489d-b4a9-f116c0b0d388 service neutron] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:00.799827 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f395cc-4add-489d-b4a9-f116c0b0d388 service neutron] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:00.800102 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f395cc-4add-489d-b4a9-f116c0b0d388 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:00.802228 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0634de9-3317-443c-99a9-c0c81cfa0ec3 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:00.802579 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0634de9-3317-443c-99a9-c0c81cfa0ec3 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:00.802977 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0634de9-3317-443c-99a9-c0c81cfa0ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:00.811175 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f395cc-4add-489d-b4a9-f116c0b0d388 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:00.812810 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f395cc-4add-489d-b4a9-f116c0b0d388 service neutron] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:00.824686 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 810/1626] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:14:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:14:00.825537 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f395cc-4add-489d-b4a9-f116c0b0d388 service neutron] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:00.841180 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f395cc-4add-489d-b4a9-f116c0b0d388 service neutron] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:00.845011 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480d6d05-3e8d-4911-a7f7-2faecc86bd6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:00.845655 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480d6d05-3e8d-4911-a7f7-2faecc86bd6d service cinder] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:00.845955 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480d6d05-3e8d-4911-a7f7-2faecc86bd6d service cinder] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:00.846234 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480d6d05-3e8d-4911-a7f7-2faecc86bd6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:00.859977 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480d6d05-3e8d-4911-a7f7-2faecc86bd6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:00.861957 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480d6d05-3e8d-4911-a7f7-2faecc86bd6d service cinder] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:00.884291 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-a0634de9-3317-443c-99a9-c0c81cfa0ec3 None None] MFA Rules not processed for user `7aacd7d6d17e4f10ba5c9107a12beafa`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:00.886259 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-a0634de9-3317-443c-99a9-c0c81cfa0ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:00.886873 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 811/1627] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:14:00 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:14:00.887870 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480d6d05-3e8d-4911-a7f7-2faecc86bd6d service cinder] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:00.908596 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480d6d05-3e8d-4911-a7f7-2faecc86bd6d service cinder] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:00.914946 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d30db1-b3d4-43c0-8d86-12fdcc168ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:00.916177 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d30db1-b3d4-43c0-8d86-12fdcc168ad1 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:00.916592 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d30db1-b3d4-43c0-8d86-12fdcc168ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:00.916925 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d30db1-b3d4-43c0-8d86-12fdcc168ad1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:00.918173 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d30db1-b3d4-43c0-8d86-12fdcc168ad1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:00.920406 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d30db1-b3d4-43c0-8d86-12fdcc168ad1 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:00.928934 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:00.936714 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 812/1628] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:14:00.941191 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d30db1-b3d4-43c0-8d86-12fdcc168ad1 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:00.955624 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d30db1-b3d4-43c0-8d86-12fdcc168ad1 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:00.964466 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9734f3d-3b94-4de6-b141-ab00a60bd274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:00.965272 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9734f3d-3b94-4de6-b141-ab00a60bd274 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:00.965639 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9734f3d-3b94-4de6-b141-ab00a60bd274 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:00.966334 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9734f3d-3b94-4de6-b141-ab00a60bd274 admin admin] PATH_INFO: `/v3/users/77ba5afd671e4b2b95470d20c170651e` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:00.968042 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9734f3d-3b94-4de6-b141-ab00a60bd274 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77ba5afd671e4b2b95470d20c170651e)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:00.971237 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9734f3d-3b94-4de6-b141-ab00a60bd274 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:01.028728 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 817/1629] 10.222.0.61 () {58 vars in 1059 bytes} [Fri Aug 26 12:14:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:01.030512 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0634de9-3317-443c-99a9-c0c81cfa0ec3 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:01.045571 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0634de9-3317-443c-99a9-c0c81cfa0ec3 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:01.051684 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc251d6-912e-48d2-8d05-83aace9d8cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:01.052722 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc251d6-912e-48d2-8d05-83aace9d8cef admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:01.053005 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc251d6-912e-48d2-8d05-83aace9d8cef admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:01.054134 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc251d6-912e-48d2-8d05-83aace9d8cef admin admin] PATH_INFO: `/v3/projects/e51d6377f5134fb0baf62d1cf6184658/users/8dac7190e3a7464cb34eb21a6ac81de3/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:01.084652 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-f9734f3d-3b94-4de6-b141-ab00a60bd274 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77ba5afd671e4b2b95470d20c170651e'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:01.089637 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc251d6-912e-48d2-8d05-83aace9d8cef admin admin] RBAC: Authorizing `identity:create_grant(project_id=e51d6377f5134fb0baf62d1cf6184658, user_id=8dac7190e3a7464cb34eb21a6ac81de3, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:01.093840 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc251d6-912e-48d2-8d05-83aace9d8cef admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:01.104353 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-f9734f3d-3b94-4de6-b141-ab00a60bd274 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77ba5afd671e4b2b95470d20c170651e'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:01.135628 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-f9734f3d-3b94-4de6-b141-ab00a60bd274 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77ba5afd671e4b2b95470d20c170651e'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:01.137681 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-f9734f3d-3b94-4de6-b141-ab00a60bd274 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77ba5afd671e4b2b95470d20c170651e'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:01.139846 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 818/1630] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:14:01 2022] PUT /identity/v3/projects/e51d6377f5134fb0baf62d1cf6184658/users/8dac7190e3a7464cb34eb21a6ac81de3/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:01.145827 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc251d6-912e-48d2-8d05-83aace9d8cef admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:01.156851 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:01.172804 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-f9734f3d-3b94-4de6-b141-ab00a60bd274 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77ba5afd671e4b2b95470d20c170651e'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:01.190608 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 813/1631] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:14:00 2022] DELETE /identity/v3/users/77ba5afd671e4b2b95470d20c170651e => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:01.191318 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9734f3d-3b94-4de6-b141-ab00a60bd274 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:01.195870 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-f9734f3d-3b94-4de6-b141-ab00a60bd274 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:01.351768 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc251d6-912e-48d2-8d05-83aace9d8cef admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:01.358901 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9734f3d-3b94-4de6-b141-ab00a60bd274 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:01.365802 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb2f55e-fe5d-4d8f-abae-53334d5e47a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:01.366522 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb2f55e-fe5d-4d8f-abae-53334d5e47a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:01.366881 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb2f55e-fe5d-4d8f-abae-53334d5e47a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:01.367274 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb2f55e-fe5d-4d8f-abae-53334d5e47a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:01.369460 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb2f55e-fe5d-4d8f-abae-53334d5e47a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:01.371574 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb2f55e-fe5d-4d8f-abae-53334d5e47a2 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:01.386169 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 814/1632] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:14:01.388320 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33f72b5-e889-4142-871b-b6d41007e5a4 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:01.388672 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33f72b5-e889-4142-871b-b6d41007e5a4 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:01.388970 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33f72b5-e889-4142-871b-b6d41007e5a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:01.413570 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-d33f72b5-e889-4142-871b-b6d41007e5a4 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:01.414802 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-d33f72b5-e889-4142-871b-b6d41007e5a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:01.425584 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 815/1633] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:01.427121 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33f72b5-e889-4142-871b-b6d41007e5a4 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:01.440652 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33f72b5-e889-4142-871b-b6d41007e5a4 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:01.471425 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8b0bb-dcf3-4b84-b439-3d7d8f7f24da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:01.473278 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8b0bb-dcf3-4b84-b439-3d7d8f7f24da admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:01.473515 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8b0bb-dcf3-4b84-b439-3d7d8f7f24da admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:01.473826 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8b0bb-dcf3-4b84-b439-3d7d8f7f24da admin admin] PATH_INFO: `/v3/projects/e51d6377f5134fb0baf62d1cf6184658/users/8dac7190e3a7464cb34eb21a6ac81de3/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:01.476481 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c8b0bb-dcf3-4b84-b439-3d7d8f7f24da admin admin] RBAC: Authorizing `identity:create_grant(project_id=e51d6377f5134fb0baf62d1cf6184658, user_id=8dac7190e3a7464cb34eb21a6ac81de3, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:01.482582 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47b1cb2-9384-4b83-8c32-38154442cce9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aacd7d6d17e4f10ba5c9107a12beafa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:01.483773 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47b1cb2-9384-4b83-8c32-38154442cce9 service nova] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:01.484399 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47b1cb2-9384-4b83-8c32-38154442cce9 service nova] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:01.485234 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47b1cb2-9384-4b83-8c32-38154442cce9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:01.485912 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c8b0bb-dcf3-4b84-b439-3d7d8f7f24da admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:01.488453 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-b47b1cb2-9384-4b83-8c32-38154442cce9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:01.522872 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 816/1634] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:14:01 2022] PUT /identity/v3/projects/e51d6377f5134fb0baf62d1cf6184658/users/8dac7190e3a7464cb34eb21a6ac81de3/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:01.524169 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8b0bb-dcf3-4b84-b439-3d7d8f7f24da admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:01.529637 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-b9c8b0bb-dcf3-4b84-b439-3d7d8f7f24da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:01.557252 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:01.590899 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47b1cb2-9384-4b83-8c32-38154442cce9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:01.593289 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47b1cb2-9384-4b83-8c32-38154442cce9 service nova] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:01.605634 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 819/1635] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:14:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:14:01.607281 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592273fe-76f0-468c-9913-6db1bf3301a0 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:01.607674 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592273fe-76f0-468c-9913-6db1bf3301a0 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:01.608595 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592273fe-76f0-468c-9913-6db1bf3301a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:01.653597 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-592273fe-76f0-468c-9913-6db1bf3301a0 None None] MFA Rules not processed for user `8dac7190e3a7464cb34eb21a6ac81de3`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:01.655062 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-592273fe-76f0-468c-9913-6db1bf3301a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:01.768374 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8b0bb-dcf3-4b84-b439-3d7d8f7f24da admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:01.773334 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf77b22-54d0-4043-8cd4-078c93921cd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:01.774174 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf77b22-54d0-4043-8cd4-078c93921cd2 service neutron] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:01.774767 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf77b22-54d0-4043-8cd4-078c93921cd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:01.775396 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf77b22-54d0-4043-8cd4-078c93921cd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:01.803672 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 820/1636] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:01 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:01.816351 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592273fe-76f0-468c-9913-6db1bf3301a0 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:01.819989 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-592273fe-76f0-468c-9913-6db1bf3301a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:01.924357 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592273fe-76f0-468c-9913-6db1bf3301a0 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:01.928387 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9783aa39-ae50-4170-b357-53256010f187 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:01.929053 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9783aa39-ae50-4170-b357-53256010f187 service cinder] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:01.929339 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9783aa39-ae50-4170-b357-53256010f187 service cinder] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:01.929611 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9783aa39-ae50-4170-b357-53256010f187 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:01.940929 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9783aa39-ae50-4170-b357-53256010f187 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:01.942431 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9783aa39-ae50-4170-b357-53256010f187 service cinder] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:01.959913 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 821/1637] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:14:01 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:14:02.016068 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf77b22-54d0-4043-8cd4-078c93921cd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:02.017799 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf77b22-54d0-4043-8cd4-078c93921cd2 service neutron] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:02.044921 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 817/1638] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:14:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:14:02.438240 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9783aa39-ae50-4170-b357-53256010f187 service cinder] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:02.459530 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9783aa39-ae50-4170-b357-53256010f187 service cinder] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:02.472456 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f75365f-02c7-4ca1-bb56-5ced6a505c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:02.472682 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75365f-02c7-4ca1-bb56-5ced6a505c4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:02.473012 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75365f-02c7-4ca1-bb56-5ced6a505c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:02.473590 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75365f-02c7-4ca1-bb56-5ced6a505c4f admin admin] PATH_INFO: `/v3/users/be2144973da4484b934173964168da80` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:02.476863 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f75365f-02c7-4ca1-bb56-5ced6a505c4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=be2144973da4484b934173964168da80)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:02.478907 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: WARNING py.warnings [None req-3f75365f-02c7-4ca1-bb56-5ced6a505c4f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 12:14:02.478907 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: warnings.warn(msg) Aug 26 12:14:02.478907 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: Aug 26 12:14:02.480399 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f75365f-02c7-4ca1-bb56-5ced6a505c4f admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:02.629741 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-3f75365f-02c7-4ca1-bb56-5ced6a505c4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be2144973da4484b934173964168da80'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:02.652196 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-3f75365f-02c7-4ca1-bb56-5ced6a505c4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be2144973da4484b934173964168da80'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:02.652972 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-3f75365f-02c7-4ca1-bb56-5ced6a505c4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be2144973da4484b934173964168da80'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:02.657951 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:02.666463 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68c6b49-d157-4b99-a192-c5ac2e89036c None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:02.667052 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68c6b49-d157-4b99-a192-c5ac2e89036c None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:02.668115 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68c6b49-d157-4b99-a192-c5ac2e89036c None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:02.670459 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 818/1639] 10.222.0.61 () {54 vars in 962 bytes} [Fri Aug 26 12:14:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:14:02.675439 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-3f75365f-02c7-4ca1-bb56-5ced6a505c4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be2144973da4484b934173964168da80'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:02.677305 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-3f75365f-02c7-4ca1-bb56-5ced6a505c4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be2144973da4484b934173964168da80'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:02.683949 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd6f50b-cae7-4992-b7ac-fbf974fcaf63 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:02.684500 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd6f50b-cae7-4992-b7ac-fbf974fcaf63 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:02.684971 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd6f50b-cae7-4992-b7ac-fbf974fcaf63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:02.742122 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 822/1640] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:14:02 2022] DELETE /identity/v3/users/be2144973da4484b934173964168da80 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:02.749477 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0039f294-1200-4d88-b0f5-4636e2ff38f9 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:02.750009 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0039f294-1200-4d88-b0f5-4636e2ff38f9 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:02.750475 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0039f294-1200-4d88-b0f5-4636e2ff38f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:02.792019 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-9dd6f50b-cae7-4992-b7ac-fbf974fcaf63 None None] MFA Rules not processed for user `7aacd7d6d17e4f10ba5c9107a12beafa`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:02.795816 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-0039f294-1200-4d88-b0f5-4636e2ff38f9 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:02.796221 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-9dd6f50b-cae7-4992-b7ac-fbf974fcaf63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:02.797290 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-0039f294-1200-4d88-b0f5-4636e2ff38f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:02.813319 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:02.950240 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 823/1641] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:02.960364 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 819/1642] 10.222.0.61 () {58 vars in 1059 bytes} [Fri Aug 26 12:14:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:02.967235 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0039f294-1200-4d88-b0f5-4636e2ff38f9 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:02.969052 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-0039f294-1200-4d88-b0f5-4636e2ff38f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:02.971585 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7c900e-c555-4211-bbba-7c9a9574a084 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:02.971872 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7c900e-c555-4211-bbba-7c9a9574a084 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:02.972143 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7c900e-c555-4211-bbba-7c9a9574a084 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:02.978407 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 820/1643] 10.222.0.61 () {54 vars in 912 bytes} [Fri Aug 26 12:14:02 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:14:02.981724 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd29021-f27a-4ffb-9d6c-db2a36164346 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:02.981724 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd29021-f27a-4ffb-9d6c-db2a36164346 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:02.981724 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd29021-f27a-4ffb-9d6c-db2a36164346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:03.044930 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-8bd29021-f27a-4ffb-9d6c-db2a36164346 None None] MFA Rules not processed for user `7aacd7d6d17e4f10ba5c9107a12beafa`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:03.046688 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-8bd29021-f27a-4ffb-9d6c-db2a36164346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:03.070211 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 821/1644] 10.222.0.61 () {58 vars in 1009 bytes} [Fri Aug 26 12:14:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:03.080273 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0039f294-1200-4d88-b0f5-4636e2ff38f9 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:03.089885 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37dc39b-c9e1-4f61-a6cb-0c2fc832e1ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:03.090513 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd29021-f27a-4ffb-9d6c-db2a36164346 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:03.091233 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37dc39b-c9e1-4f61-a6cb-0c2fc832e1ae service neutron] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:03.091849 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37dc39b-c9e1-4f61-a6cb-0c2fc832e1ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:03.092175 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-8bd29021-f27a-4ffb-9d6c-db2a36164346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:03.093056 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37dc39b-c9e1-4f61-a6cb-0c2fc832e1ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:03.110739 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37dc39b-c9e1-4f61-a6cb-0c2fc832e1ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:03.112869 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37dc39b-c9e1-4f61-a6cb-0c2fc832e1ae service neutron] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:03.132210 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 824/1645] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:14:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 12:14:03.173417 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37dc39b-c9e1-4f61-a6cb-0c2fc832e1ae service neutron] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:03.175408 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-b37dc39b-c9e1-4f61-a6cb-0c2fc832e1ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:03.222349 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd29021-f27a-4ffb-9d6c-db2a36164346 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:03.226544 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ae5496-20e0-44a4-8362-efacac2a5bb6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:03.227434 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ae5496-20e0-44a4-8362-efacac2a5bb6 service cinder] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:03.228106 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ae5496-20e0-44a4-8362-efacac2a5bb6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:03.228247 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37dc39b-c9e1-4f61-a6cb-0c2fc832e1ae service neutron] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:03.228893 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ae5496-20e0-44a4-8362-efacac2a5bb6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:03.239585 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c0c52c-80c2-4cc3-a01c-bc0fceda0f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:03.243765 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c0c52c-80c2-4cc3-a01c-bc0fceda0f97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:03.245120 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c0c52c-80c2-4cc3-a01c-bc0fceda0f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:03.245409 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c0c52c-80c2-4cc3-a01c-bc0fceda0f97 admin admin] PATH_INFO: `/v3/projects/4b937d8a156b4be0994719f3399f2d14` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:03.245999 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ae5496-20e0-44a4-8362-efacac2a5bb6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:03.247952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ae5496-20e0-44a4-8362-efacac2a5bb6 service cinder] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:03.249264 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c0c52c-80c2-4cc3-a01c-bc0fceda0f97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b937d8a156b4be0994719f3399f2d14)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:03.251101 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c0c52c-80c2-4cc3-a01c-bc0fceda0f97 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:03.265326 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 822/1646] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:14:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 12:14:03.295792 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd3537-c2c1-47e3-83b4-606a10f4eba4 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:03.295985 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd3537-c2c1-47e3-83b4-606a10f4eba4 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:03.296230 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd3537-c2c1-47e3-83b4-606a10f4eba4 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:03.303263 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 823/1647] 10.222.0.61 () {54 vars in 912 bytes} [Fri Aug 26 12:14:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:14:03.312715 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2521a506-db23-4328-a289-6a7a69e4fc24 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:03.312978 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2521a506-db23-4328-a289-6a7a69e4fc24 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:03.313250 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2521a506-db23-4328-a289-6a7a69e4fc24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:03.346005 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-2521a506-db23-4328-a289-6a7a69e4fc24 None None] MFA Rules not processed for user `7aacd7d6d17e4f10ba5c9107a12beafa`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:03.347001 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-2521a506-db23-4328-a289-6a7a69e4fc24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:03.364871 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 824/1648] 10.222.0.61 () {58 vars in 1009 bytes} [Fri Aug 26 12:14:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:03.389275 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2521a506-db23-4328-a289-6a7a69e4fc24 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:03.391896 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-2521a506-db23-4328-a289-6a7a69e4fc24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:03.409366 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2521a506-db23-4328-a289-6a7a69e4fc24 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:03.429488 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-50c0c52c-80c2-4cc3-a01c-bc0fceda0f97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b937d8a156b4be0994719f3399f2d14', 'request_id': 'req-50c0c52c-80c2-4cc3-a01c-bc0fceda0f97'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:03.445163 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-50c0c52c-80c2-4cc3-a01c-bc0fceda0f97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b937d8a156b4be0994719f3399f2d14', 'request_id': 'req-50c0c52c-80c2-4cc3-a01c-bc0fceda0f97'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:03.456148 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-50c0c52c-80c2-4cc3-a01c-bc0fceda0f97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b937d8a156b4be0994719f3399f2d14', 'request_id': 'req-50c0c52c-80c2-4cc3-a01c-bc0fceda0f97'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:03.467892 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:03.474298 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-50c0c52c-80c2-4cc3-a01c-bc0fceda0f97 admin admin] The token cache is being invalidate because project 4b937d8a156b4be0994719f3399f2d14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81712) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:14:03.474777 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-50c0c52c-80c2-4cc3-a01c-bc0fceda0f97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:03.476197 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-50c0c52c-80c2-4cc3-a01c-bc0fceda0f97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:03.512312 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 825/1649] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:14:03 2022] DELETE /identity/v3/projects/4b937d8a156b4be0994719f3399f2d14 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:03.514745 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c0c52c-80c2-4cc3-a01c-bc0fceda0f97 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:03.519984 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-50c0c52c-80c2-4cc3-a01c-bc0fceda0f97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:03.527432 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803d994d-a0a8-4dd9-8d0c-dce096853ba9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:03.527726 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d994d-a0a8-4dd9-8d0c-dce096853ba9 service cinder] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:03.527987 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d994d-a0a8-4dd9-8d0c-dce096853ba9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:03.528261 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d994d-a0a8-4dd9-8d0c-dce096853ba9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:03.537635 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-803d994d-a0a8-4dd9-8d0c-dce096853ba9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:03.632021 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c0c52c-80c2-4cc3-a01c-bc0fceda0f97 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:03.632627 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:03.641159 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a661c6e-4a28-4361-a9d5-66b8815aff9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:03.642403 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a661c6e-4a28-4361-a9d5-66b8815aff9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:03.642814 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a661c6e-4a28-4361-a9d5-66b8815aff9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:03.643187 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a661c6e-4a28-4361-a9d5-66b8815aff9e admin admin] PATH_INFO: `/v3/users/1c887b76db5f48b98a5dbe1632e7eac0` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:03.646490 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a661c6e-4a28-4361-a9d5-66b8815aff9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c887b76db5f48b98a5dbe1632e7eac0)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:03.648104 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a661c6e-4a28-4361-a9d5-66b8815aff9e admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:03.701070 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803d994d-a0a8-4dd9-8d0c-dce096853ba9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:03.703611 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803d994d-a0a8-4dd9-8d0c-dce096853ba9 service cinder] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:03.747491 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 825/1650] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:14:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:14:03.748403 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803d994d-a0a8-4dd9-8d0c-dce096853ba9 service cinder] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:03.753093 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-5a661c6e-4a28-4361-a9d5-66b8815aff9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c887b76db5f48b98a5dbe1632e7eac0'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:03.754611 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-803d994d-a0a8-4dd9-8d0c-dce096853ba9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:03.769318 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-5a661c6e-4a28-4361-a9d5-66b8815aff9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c887b76db5f48b98a5dbe1632e7eac0'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:03.770300 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-5a661c6e-4a28-4361-a9d5-66b8815aff9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c887b76db5f48b98a5dbe1632e7eac0'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:03.783673 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-5a661c6e-4a28-4361-a9d5-66b8815aff9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c887b76db5f48b98a5dbe1632e7eac0'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:03.784749 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-5a661c6e-4a28-4361-a9d5-66b8815aff9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c887b76db5f48b98a5dbe1632e7eac0'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:03.789857 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803d994d-a0a8-4dd9-8d0c-dce096853ba9 service cinder] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:03.803652 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2cf235-2d70-4700-816c-51f40f1a7d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:03.804335 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2cf235-2d70-4700-816c-51f40f1a7d2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:03.804585 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2cf235-2d70-4700-816c-51f40f1a7d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:03.804861 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2cf235-2d70-4700-816c-51f40f1a7d2e admin admin] PATH_INFO: `/v3/users/cee554500b02434c8c5aad6d498bc287` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:03.810589 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2cf235-2d70-4700-816c-51f40f1a7d2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cee554500b02434c8c5aad6d498bc287)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:03.810863 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 826/1651] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:14:03 2022] DELETE /identity/v3/users/1c887b76db5f48b98a5dbe1632e7eac0 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:03.813741 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2cf235-2d70-4700-816c-51f40f1a7d2e admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:03.901489 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-2b2cf235-2d70-4700-816c-51f40f1a7d2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cee554500b02434c8c5aad6d498bc287'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:03.919397 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-2b2cf235-2d70-4700-816c-51f40f1a7d2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cee554500b02434c8c5aad6d498bc287'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:03.930178 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-2b2cf235-2d70-4700-816c-51f40f1a7d2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cee554500b02434c8c5aad6d498bc287'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:03.930985 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-2b2cf235-2d70-4700-816c-51f40f1a7d2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cee554500b02434c8c5aad6d498bc287'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:03.959811 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-2b2cf235-2d70-4700-816c-51f40f1a7d2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cee554500b02434c8c5aad6d498bc287'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:03.971944 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 826/1652] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:14:03 2022] DELETE /identity/v3/users/cee554500b02434c8c5aad6d498bc287 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:03.979891 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30cb81-67b7-4b5b-8d9b-638c4df1fb2d None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:03.980177 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30cb81-67b7-4b5b-8d9b-638c4df1fb2d None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:03.980454 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30cb81-67b7-4b5b-8d9b-638c4df1fb2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:04.038947 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-bf30cb81-67b7-4b5b-8d9b-638c4df1fb2d None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:04.045199 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-bf30cb81-67b7-4b5b-8d9b-638c4df1fb2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:04.215603 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 827/1653] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:04.263386 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2cf235-2d70-4700-816c-51f40f1a7d2e admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:04.264775 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-2b2cf235-2d70-4700-816c-51f40f1a7d2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:04.400321 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2cf235-2d70-4700-816c-51f40f1a7d2e admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:04.404612 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db758b3-1eb3-4246-83cb-ee2e437c0320 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:04.405568 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db758b3-1eb3-4246-83cb-ee2e437c0320 service neutron] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:04.406096 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db758b3-1eb3-4246-83cb-ee2e437c0320 service neutron] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:04.406574 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db758b3-1eb3-4246-83cb-ee2e437c0320 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:04.419039 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db758b3-1eb3-4246-83cb-ee2e437c0320 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:04.421102 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db758b3-1eb3-4246-83cb-ee2e437c0320 service neutron] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:04.472565 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 827/1654] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:14:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:14:05.639872 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf30cb81-67b7-4b5b-8d9b-638c4df1fb2d None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:05.658365 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-bf30cb81-67b7-4b5b-8d9b-638c4df1fb2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:05.682296 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf30cb81-67b7-4b5b-8d9b-638c4df1fb2d None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:05.694880 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a594fc4f-6a9c-480f-8207-fc15e7ea500a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:05.695739 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a594fc4f-6a9c-480f-8207-fc15e7ea500a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:05.696023 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a594fc4f-6a9c-480f-8207-fc15e7ea500a admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:05.696358 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a594fc4f-6a9c-480f-8207-fc15e7ea500a admin admin] PATH_INFO: `/v3/projects/5a81358b458943c3abac971c3dcf9636` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:05.697917 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a594fc4f-6a9c-480f-8207-fc15e7ea500a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a81358b458943c3abac971c3dcf9636)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:05.699966 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a594fc4f-6a9c-480f-8207-fc15e7ea500a admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:05.746524 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:05.834490 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-a594fc4f-6a9c-480f-8207-fc15e7ea500a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a81358b458943c3abac971c3dcf9636', 'request_id': 'req-a594fc4f-6a9c-480f-8207-fc15e7ea500a'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:05.847169 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-a594fc4f-6a9c-480f-8207-fc15e7ea500a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a81358b458943c3abac971c3dcf9636', 'request_id': 'req-a594fc4f-6a9c-480f-8207-fc15e7ea500a'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:05.870703 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-a594fc4f-6a9c-480f-8207-fc15e7ea500a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a81358b458943c3abac971c3dcf9636', 'request_id': 'req-a594fc4f-6a9c-480f-8207-fc15e7ea500a'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:05.892306 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-a594fc4f-6a9c-480f-8207-fc15e7ea500a admin admin] The token cache is being invalidate because project 5a81358b458943c3abac971c3dcf9636 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81712) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:14:05.892306 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-a594fc4f-6a9c-480f-8207-fc15e7ea500a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:05.892952 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-a594fc4f-6a9c-480f-8207-fc15e7ea500a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:05.894780 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 828/1655] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:14:05 2022] DELETE /identity/v3/projects/5a81358b458943c3abac971c3dcf9636 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:05.898668 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db758b3-1eb3-4246-83cb-ee2e437c0320 service neutron] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:05.918024 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db758b3-1eb3-4246-83cb-ee2e437c0320 service neutron] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:05.970296 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a594fc4f-6a9c-480f-8207-fc15e7ea500a admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:05.971780 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-a594fc4f-6a9c-480f-8207-fc15e7ea500a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:06.080604 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:06.111864 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882905c6-70ea-483b-832e-9467965c7e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:06.112919 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882905c6-70ea-483b-832e-9467965c7e6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:06.113887 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882905c6-70ea-483b-832e-9467965c7e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:06.120603 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882905c6-70ea-483b-832e-9467965c7e6e admin admin] PATH_INFO: `/v3/users/8dac7190e3a7464cb34eb21a6ac81de3` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:06.144320 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882905c6-70ea-483b-832e-9467965c7e6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8dac7190e3a7464cb34eb21a6ac81de3)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:06.147942 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882905c6-70ea-483b-832e-9467965c7e6e admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:06.240143 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a594fc4f-6a9c-480f-8207-fc15e7ea500a admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:06.297488 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e05a93a-fcf0-4d73-b746-c4296e125ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:06.298250 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e05a93a-fcf0-4d73-b746-c4296e125ac5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:06.298986 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e05a93a-fcf0-4d73-b746-c4296e125ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:06.299487 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e05a93a-fcf0-4d73-b746-c4296e125ac5 admin admin] PATH_INFO: `/v3/projects/a0e89065424a4994842f3469465e394c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:06.301973 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e05a93a-fcf0-4d73-b746-c4296e125ac5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0e89065424a4994842f3469465e394c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:06.310523 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e05a93a-fcf0-4d73-b746-c4296e125ac5 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:06.424681 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-8e05a93a-fcf0-4d73-b746-c4296e125ac5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0e89065424a4994842f3469465e394c', 'request_id': 'req-8e05a93a-fcf0-4d73-b746-c4296e125ac5'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:06.452193 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-8e05a93a-fcf0-4d73-b746-c4296e125ac5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0e89065424a4994842f3469465e394c', 'request_id': 'req-8e05a93a-fcf0-4d73-b746-c4296e125ac5'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:06.458130 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-882905c6-70ea-483b-832e-9467965c7e6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8dac7190e3a7464cb34eb21a6ac81de3'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:06.475602 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-8e05a93a-fcf0-4d73-b746-c4296e125ac5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0e89065424a4994842f3469465e394c', 'request_id': 'req-8e05a93a-fcf0-4d73-b746-c4296e125ac5'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:06.496281 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-882905c6-70ea-483b-832e-9467965c7e6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8dac7190e3a7464cb34eb21a6ac81de3'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:06.504694 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-8e05a93a-fcf0-4d73-b746-c4296e125ac5 admin admin] The token cache is being invalidate because project a0e89065424a4994842f3469465e394c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81712) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:14:06.504997 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-8e05a93a-fcf0-4d73-b746-c4296e125ac5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:06.505800 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-8e05a93a-fcf0-4d73-b746-c4296e125ac5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:06.506601 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-882905c6-70ea-483b-832e-9467965c7e6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8dac7190e3a7464cb34eb21a6ac81de3'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:06.507494 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-882905c6-70ea-483b-832e-9467965c7e6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8dac7190e3a7464cb34eb21a6ac81de3'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:06.508313 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 829/1656] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:14:05 2022] DELETE /identity/v3/projects/a0e89065424a4994842f3469465e394c => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:06.512813 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e05a93a-fcf0-4d73-b746-c4296e125ac5 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:06.515607 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-8e05a93a-fcf0-4d73-b746-c4296e125ac5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:06.553492 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-882905c6-70ea-483b-832e-9467965c7e6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8dac7190e3a7464cb34eb21a6ac81de3'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:06.598501 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 828/1657] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:14:05 2022] DELETE /identity/v3/users/8dac7190e3a7464cb34eb21a6ac81de3 => generated 0 bytes in 700 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:06.604804 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f96c84-67c4-4701-9bf4-070c423e26d2 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:06.605411 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f96c84-67c4-4701-9bf4-070c423e26d2 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:06.605788 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f96c84-67c4-4701-9bf4-070c423e26d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:06.617001 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:06.686903 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-25f96c84-67c4-4701-9bf4-070c423e26d2 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:06.688929 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-25f96c84-67c4-4701-9bf4-070c423e26d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:06.760214 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e05a93a-fcf0-4d73-b746-c4296e125ac5 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:07.023336 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 829/1658] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:07.038855 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e4d3c-b790-4ced-be59-ae6fdb362653 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:07.039570 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e4d3c-b790-4ced-be59-ae6fdb362653 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:07.040032 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e4d3c-b790-4ced-be59-ae6fdb362653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:07.047561 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14f5a07-cf35-46c7-a74e-7ed6ee35ce71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:07.048648 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14f5a07-cf35-46c7-a74e-7ed6ee35ce71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:07.049782 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14f5a07-cf35-46c7-a74e-7ed6ee35ce71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:07.051487 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14f5a07-cf35-46c7-a74e-7ed6ee35ce71 admin admin] PATH_INFO: `/v3/projects/aff0bc2a1f414ae2ad59f54e2bb5ff36` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:07.055613 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14f5a07-cf35-46c7-a74e-7ed6ee35ce71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aff0bc2a1f414ae2ad59f54e2bb5ff36)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:07.063157 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14f5a07-cf35-46c7-a74e-7ed6ee35ce71 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:07.104741 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-d75e4d3c-b790-4ced-be59-ae6fdb362653 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:07.106217 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-d75e4d3c-b790-4ced-be59-ae6fdb362653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:07.135859 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 830/1659] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:07.139779 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e4d3c-b790-4ced-be59-ae6fdb362653 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:07.170643 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e4d3c-b790-4ced-be59-ae6fdb362653 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:07.186744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbe28de-7108-49af-8018-c7bed7b70fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:07.187470 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe28de-7108-49af-8018-c7bed7b70fc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:07.187759 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe28de-7108-49af-8018-c7bed7b70fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:07.188030 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe28de-7108-49af-8018-c7bed7b70fc0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:07.188749 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbe28de-7108-49af-8018-c7bed7b70fc0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:14:07.188997 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbe28de-7108-49af-8018-c7bed7b70fc0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:07.190889 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbe28de-7108-49af-8018-c7bed7b70fc0 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:07.232522 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 831/1660] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:14:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:14:07.233906 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbe28de-7108-49af-8018-c7bed7b70fc0 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:07.251742 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbe28de-7108-49af-8018-c7bed7b70fc0 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:07.253261 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-f14f5a07-cf35-46c7-a74e-7ed6ee35ce71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aff0bc2a1f414ae2ad59f54e2bb5ff36', 'request_id': 'req-f14f5a07-cf35-46c7-a74e-7ed6ee35ce71'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:07.282696 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-f14f5a07-cf35-46c7-a74e-7ed6ee35ce71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aff0bc2a1f414ae2ad59f54e2bb5ff36', 'request_id': 'req-f14f5a07-cf35-46c7-a74e-7ed6ee35ce71'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:07.301305 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-f14f5a07-cf35-46c7-a74e-7ed6ee35ce71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aff0bc2a1f414ae2ad59f54e2bb5ff36', 'request_id': 'req-f14f5a07-cf35-46c7-a74e-7ed6ee35ce71'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:07.332165 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-f14f5a07-cf35-46c7-a74e-7ed6ee35ce71 admin admin] The token cache is being invalidate because project aff0bc2a1f414ae2ad59f54e2bb5ff36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81712) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:14:07.332458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-f14f5a07-cf35-46c7-a74e-7ed6ee35ce71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:07.339563 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-f14f5a07-cf35-46c7-a74e-7ed6ee35ce71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:07.345267 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 830/1661] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:14:06 2022] DELETE /identity/v3/projects/aff0bc2a1f414ae2ad59f54e2bb5ff36 => generated 0 bytes in 836 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:07.352610 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a334daf1-3bc4-47fc-a39d-600b32f4f718 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:07.352900 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a334daf1-3bc4-47fc-a39d-600b32f4f718 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:07.358050 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a334daf1-3bc4-47fc-a39d-600b32f4f718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:07.374235 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:07.402217 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7714b54b-ad2a-451f-b14e-f5f6308b59ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:07.407966 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7714b54b-ad2a-451f-b14e-f5f6308b59ac admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:07.408255 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7714b54b-ad2a-451f-b14e-f5f6308b59ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:07.408601 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7714b54b-ad2a-451f-b14e-f5f6308b59ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:07.409260 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7714b54b-ad2a-451f-b14e-f5f6308b59ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:07.411423 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7714b54b-ad2a-451f-b14e-f5f6308b59ac admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:07.436360 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-a334daf1-3bc4-47fc-a39d-600b32f4f718 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:07.437613 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-a334daf1-3bc4-47fc-a39d-600b32f4f718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:07.470986 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 831/1662] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:07.472545 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beca623-6af8-4829-b7af-c2236ea692c8 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:07.472882 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beca623-6af8-4829-b7af-c2236ea692c8 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:07.473083 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beca623-6af8-4829-b7af-c2236ea692c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:07.503915 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 832/1663] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:14:07 2022] POST /identity/v3/projects => generated 387 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 12:14:07.505618 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7714b54b-ad2a-451f-b14e-f5f6308b59ac admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:07.510420 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-7714b54b-ad2a-451f-b14e-f5f6308b59ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:07.521962 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-3beca623-6af8-4829-b7af-c2236ea692c8 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:07.527370 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-3beca623-6af8-4829-b7af-c2236ea692c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:07.754969 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 832/1664] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:07.759579 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beca623-6af8-4829-b7af-c2236ea692c8 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:07.772774 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:07.789296 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7714b54b-ad2a-451f-b14e-f5f6308b59ac admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:07.793731 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beca623-6af8-4829-b7af-c2236ea692c8 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:07.814322 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df64d29-b720-449c-923d-051926dfe00e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:07.831361 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df64d29-b720-449c-923d-051926dfe00e admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:07.831602 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df64d29-b720-449c-923d-051926dfe00e admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:07.831882 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df64d29-b720-449c-923d-051926dfe00e admin admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:07.831985 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337e83bd-ecea-43ed-ba19-d8e23f7db25b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:07.832572 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df64d29-b720-449c-923d-051926dfe00e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:14:07.832879 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df64d29-b720-449c-923d-051926dfe00e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:07.834509 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df64d29-b720-449c-923d-051926dfe00e admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:07.836232 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337e83bd-ecea-43ed-ba19-d8e23f7db25b admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:07.836557 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337e83bd-ecea-43ed-ba19-d8e23f7db25b admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:07.836950 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337e83bd-ecea-43ed-ba19-d8e23f7db25b admin admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:07.840718 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337e83bd-ecea-43ed-ba19-d8e23f7db25b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:07.845079 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337e83bd-ecea-43ed-ba19-d8e23f7db25b admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:07.864541 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 833/1665] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:14:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:14:07.865773 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df64d29-b720-449c-923d-051926dfe00e admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:07.891255 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df64d29-b720-449c-923d-051926dfe00e admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:07.904289 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 833/1666] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:14:07 2022] POST /identity/v3/users => generated 294 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 12:14:07.905251 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337e83bd-ecea-43ed-ba19-d8e23f7db25b admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:07.907274 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9136a2-a39b-404a-8600-0a75ec303ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:07.908398 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9136a2-a39b-404a-8600-0a75ec303ecd admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:07.910109 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9136a2-a39b-404a-8600-0a75ec303ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:07.911675 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9136a2-a39b-404a-8600-0a75ec303ecd admin admin] PATH_INFO: `/v3/projects` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:07.913933 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9136a2-a39b-404a-8600-0a75ec303ecd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:07.922066 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9136a2-a39b-404a-8600-0a75ec303ecd admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:07.942352 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337e83bd-ecea-43ed-ba19-d8e23f7db25b admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:07.959319 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29388c71-fe89-4f95-9ab1-4686298420bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:07.960189 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29388c71-fe89-4f95-9ab1-4686298420bd admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:07.962910 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29388c71-fe89-4f95-9ab1-4686298420bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:07.963383 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29388c71-fe89-4f95-9ab1-4686298420bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:07.964297 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29388c71-fe89-4f95-9ab1-4686298420bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:07.969772 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29388c71-fe89-4f95-9ab1-4686298420bd admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:08.005077 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 834/1667] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:14:07 2022] POST /identity/v3/projects => generated 393 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:14:08.007752 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 834/1668] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:14:07 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:14:08.009921 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9136a2-a39b-404a-8600-0a75ec303ecd admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:08.019256 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29388c71-fe89-4f95-9ab1-4686298420bd admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:08.058134 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29388c71-fe89-4f95-9ab1-4686298420bd admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:08.241458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9136a2-a39b-404a-8600-0a75ec303ecd admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:08.267493 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd97f445-d8f1-4e40-965d-9fe357251dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:08.268123 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97f445-d8f1-4e40-965d-9fe357251dee admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:08.268401 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97f445-d8f1-4e40-965d-9fe357251dee admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:08.268740 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97f445-d8f1-4e40-965d-9fe357251dee admin admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:08.269306 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd97f445-d8f1-4e40-965d-9fe357251dee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:08.271280 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd97f445-d8f1-4e40-965d-9fe357251dee admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:08.286179 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1478e9d4-ec2e-4a06-92dd-6a985621191b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:08.287017 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1478e9d4-ec2e-4a06-92dd-6a985621191b admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:08.287320 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1478e9d4-ec2e-4a06-92dd-6a985621191b admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:08.290668 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1478e9d4-ec2e-4a06-92dd-6a985621191b admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:08.295258 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1478e9d4-ec2e-4a06-92dd-6a985621191b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:08.301059 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1478e9d4-ec2e-4a06-92dd-6a985621191b admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:08.355996 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 835/1669] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:14:08 2022] POST /identity/v3/users => generated 297 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 12:14:08.356488 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 835/1670] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:14:08 2022] GET /identity/v3/roles => generated 1420 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:14:08.361746 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd97f445-d8f1-4e40-965d-9fe357251dee admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:08.362083 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1478e9d4-ec2e-4a06-92dd-6a985621191b admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:08.383540 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd97f445-d8f1-4e40-965d-9fe357251dee admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:08.389753 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1478e9d4-ec2e-4a06-92dd-6a985621191b admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:08.397037 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45ae73b-ea16-476b-a972-3b2c6b85f953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:08.397726 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ae73b-ea16-476b-a972-3b2c6b85f953 admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:08.398168 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ae73b-ea16-476b-a972-3b2c6b85f953 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:08.398504 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ae73b-ea16-476b-a972-3b2c6b85f953 admin admin] PATH_INFO: `/v3/projects/fcfa7283ecf84b41ac3725ea099b4835/users/a00230e9e90448a5ad1261ccd67e8d2f/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:08.410890 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d400f0e-767d-407f-885c-6ea3ab595e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:08.411860 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d400f0e-767d-407f-885c-6ea3ab595e39 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:08.412356 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d400f0e-767d-407f-885c-6ea3ab595e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:08.412834 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d400f0e-767d-407f-885c-6ea3ab595e39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:08.413699 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d400f0e-767d-407f-885c-6ea3ab595e39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:08.415805 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d400f0e-767d-407f-885c-6ea3ab595e39 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:08.434136 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45ae73b-ea16-476b-a972-3b2c6b85f953 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcfa7283ecf84b41ac3725ea099b4835, user_id=a00230e9e90448a5ad1261ccd67e8d2f, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:08.438686 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45ae73b-ea16-476b-a972-3b2c6b85f953 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:08.461156 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 836/1671] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:14:08 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 12:14:08.465025 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d400f0e-767d-407f-885c-6ea3ab595e39 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:08.472103 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 836/1672] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:14:08 2022] PUT /identity/v3/projects/fcfa7283ecf84b41ac3725ea099b4835/users/a00230e9e90448a5ad1261ccd67e8d2f/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:08.485142 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc605024-3c78-43df-ac4a-3a839a68051f None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:08.486225 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc605024-3c78-43df-ac4a-3a839a68051f None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:08.486225 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc605024-3c78-43df-ac4a-3a839a68051f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:08.486390 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d400f0e-767d-407f-885c-6ea3ab595e39 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:08.569460 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-bc605024-3c78-43df-ac4a-3a839a68051f None None] MFA Rules not processed for user `a00230e9e90448a5ad1261ccd67e8d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:08.576484 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-bc605024-3c78-43df-ac4a-3a839a68051f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:08.590649 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:08.667800 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc92aea7-5cd7-4143-bf17-c9432f3d4099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:08.668562 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc92aea7-5cd7-4143-bf17-c9432f3d4099 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:08.668923 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc92aea7-5cd7-4143-bf17-c9432f3d4099 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:08.669265 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc92aea7-5cd7-4143-bf17-c9432f3d4099 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:08.670106 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc92aea7-5cd7-4143-bf17-c9432f3d4099 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:08.671855 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc92aea7-5cd7-4143-bf17-c9432f3d4099 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:08.706375 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 837/1673] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:14:08 2022] GET /identity/v3/roles => generated 1420 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 12:14:08.713999 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc92aea7-5cd7-4143-bf17-c9432f3d4099 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:08.762252 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 837/1674] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:08 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:14:08.763608 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc92aea7-5cd7-4143-bf17-c9432f3d4099 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:08.764036 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc605024-3c78-43df-ac4a-3a839a68051f None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:08.776175 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b01f8b-8f27-47ff-a10b-f363d1f116b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:08.776874 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b01f8b-8f27-47ff-a10b-f363d1f116b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:08.777159 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b01f8b-8f27-47ff-a10b-f363d1f116b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:08.787073 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b01f8b-8f27-47ff-a10b-f363d1f116b9 admin admin] PATH_INFO: `/v3/projects/91f90a3555f04ec9b66a36f515926155/users/1e65951661b24f0abdae7458009becfb/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:08.790742 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc605024-3c78-43df-ac4a-3a839a68051f None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:08.804814 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2ee35b-5429-48f6-8242-dc0327da1761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:08.805447 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2ee35b-5429-48f6-8242-dc0327da1761 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:08.805961 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2ee35b-5429-48f6-8242-dc0327da1761 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:08.806061 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2ee35b-5429-48f6-8242-dc0327da1761 admin admin] PATH_INFO: `/v3/projects` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:08.806682 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2ee35b-5429-48f6-8242-dc0327da1761 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:08.814034 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2ee35b-5429-48f6-8242-dc0327da1761 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:08.823388 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b01f8b-8f27-47ff-a10b-f363d1f116b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91f90a3555f04ec9b66a36f515926155, user_id=1e65951661b24f0abdae7458009becfb, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:08.826110 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b01f8b-8f27-47ff-a10b-f363d1f116b9 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:08.888237 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 838/1675] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:14:08 2022] PUT /identity/v3/projects/91f90a3555f04ec9b66a36f515926155/users/1e65951661b24f0abdae7458009becfb/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:08.902916 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f24cb7-ec8b-410c-aa7f-56c436ec838e None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:08.903568 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 838/1676] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:14:08 2022] POST /identity/v3/projects => generated 385 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:14:08.903645 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f24cb7-ec8b-410c-aa7f-56c436ec838e None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:08.903645 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f24cb7-ec8b-410c-aa7f-56c436ec838e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:08.905181 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2ee35b-5429-48f6-8242-dc0327da1761 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:08.910853 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:09.000155 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-a1f24cb7-ec8b-410c-aa7f-56c436ec838e None None] MFA Rules not processed for user `1e65951661b24f0abdae7458009becfb`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:09.001409 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-a1f24cb7-ec8b-410c-aa7f-56c436ec838e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:09.104182 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2ee35b-5429-48f6-8242-dc0327da1761 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:09.123482 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24827f4b-8c9e-4192-898c-40a3b1b34297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:09.124114 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24827f4b-8c9e-4192-898c-40a3b1b34297 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:09.124399 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24827f4b-8c9e-4192-898c-40a3b1b34297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:09.124673 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24827f4b-8c9e-4192-898c-40a3b1b34297 admin admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:09.125326 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24827f4b-8c9e-4192-898c-40a3b1b34297 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:09.127311 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24827f4b-8c9e-4192-898c-40a3b1b34297 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:09.191554 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 839/1677] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:14:08 2022] POST /identity/v3/users => generated 299 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:14:09.195495 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24827f4b-8c9e-4192-898c-40a3b1b34297 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:09.210927 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24827f4b-8c9e-4192-898c-40a3b1b34297 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:09.213311 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 839/1678] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:08 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:09.217619 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f24cb7-ec8b-410c-aa7f-56c436ec838e None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:09.235607 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95319bc-f71d-4012-9271-933b93e9527e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:09.236418 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95319bc-f71d-4012-9271-933b93e9527e admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:09.236707 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95319bc-f71d-4012-9271-933b93e9527e admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:09.237001 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95319bc-f71d-4012-9271-933b93e9527e admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:09.237648 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95319bc-f71d-4012-9271-933b93e9527e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:09.239322 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95319bc-f71d-4012-9271-933b93e9527e admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:09.252261 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f24cb7-ec8b-410c-aa7f-56c436ec838e None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:09.262956 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 840/1679] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:14:09 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 12:14:09.268977 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95319bc-f71d-4012-9271-933b93e9527e admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:09.288981 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a53f6c-62b9-4445-a0dd-8a644643cc4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:09.289608 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a53f6c-62b9-4445-a0dd-8a644643cc4b admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:09.289891 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a53f6c-62b9-4445-a0dd-8a644643cc4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:09.290200 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a53f6c-62b9-4445-a0dd-8a644643cc4b admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:09.290843 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a53f6c-62b9-4445-a0dd-8a644643cc4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:09.292946 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a53f6c-62b9-4445-a0dd-8a644643cc4b admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:09.319123 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95319bc-f71d-4012-9271-933b93e9527e admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:09.339968 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5440576-ca58-4d89-9b04-573423172fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:09.343573 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5440576-ca58-4d89-9b04-573423172fdd admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:09.343859 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5440576-ca58-4d89-9b04-573423172fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:09.346527 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5440576-ca58-4d89-9b04-573423172fdd admin admin] PATH_INFO: `/v3/projects/dab58d1c4b6b48d88067f316d5aa0120/users/e81659c46f6d4354a0897e4fde79d1a2/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:09.424674 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5440576-ca58-4d89-9b04-573423172fdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=dab58d1c4b6b48d88067f316d5aa0120, user_id=e81659c46f6d4354a0897e4fde79d1a2, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:09.428251 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 840/1680] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:14:09 2022] POST /identity/v3/projects => generated 391 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 12:14:09.429718 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a53f6c-62b9-4445-a0dd-8a644643cc4b admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:09.450090 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5440576-ca58-4d89-9b04-573423172fdd admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:09.518786 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 841/1681] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:14:09 2022] PUT /identity/v3/projects/dab58d1c4b6b48d88067f316d5aa0120/users/e81659c46f6d4354a0897e4fde79d1a2/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 12:14:09.525159 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5440576-ca58-4d89-9b04-573423172fdd admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:09.671842 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a53f6c-62b9-4445-a0dd-8a644643cc4b admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:09.741606 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:09.770408 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5440576-ca58-4d89-9b04-573423172fdd admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:09.810166 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2a1787-587c-4d84-9aa6-9bff52e73c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:09.810791 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2a1787-587c-4d84-9aa6-9bff52e73c3c admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:09.811383 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2a1787-587c-4d84-9aa6-9bff52e73c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:09.811383 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2a1787-587c-4d84-9aa6-9bff52e73c3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:09.812008 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2a1787-587c-4d84-9aa6-9bff52e73c3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:09.824351 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2a1787-587c-4d84-9aa6-9bff52e73c3c admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:09.853746 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04d6a6c-4d6b-405b-97f2-8ebbb7b3ca3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:09.858974 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d6a6c-4d6b-405b-97f2-8ebbb7b3ca3e admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:09.859393 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d6a6c-4d6b-405b-97f2-8ebbb7b3ca3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:09.861143 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d6a6c-4d6b-405b-97f2-8ebbb7b3ca3e admin admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:09.863218 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04d6a6c-4d6b-405b-97f2-8ebbb7b3ca3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:09.865323 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04d6a6c-4d6b-405b-97f2-8ebbb7b3ca3e admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:09.888264 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 842/1682] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:14:09 2022] GET /identity/v3/roles => generated 1420 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:14:09.892783 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2a1787-587c-4d84-9aa6-9bff52e73c3c admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:09.949840 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 841/1683] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:14:09 2022] POST /identity/v3/users => generated 302 bytes in 521 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:14:09.951849 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2a1787-587c-4d84-9aa6-9bff52e73c3c admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:09.960977 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04d6a6c-4d6b-405b-97f2-8ebbb7b3ca3e admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:09.981609 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664b2d04-05e0-4b9e-a65d-9b7817640da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:09.984420 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b2d04-05e0-4b9e-a65d-9b7817640da2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:09.985508 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b2d04-05e0-4b9e-a65d-9b7817640da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:09.985508 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b2d04-05e0-4b9e-a65d-9b7817640da2 admin admin] PATH_INFO: `/v3/projects/dab58d1c4b6b48d88067f316d5aa0120/users/e81659c46f6d4354a0897e4fde79d1a2/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:09.995679 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664b2d04-05e0-4b9e-a65d-9b7817640da2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dab58d1c4b6b48d88067f316d5aa0120, user_id=e81659c46f6d4354a0897e4fde79d1a2, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:10.002963 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664b2d04-05e0-4b9e-a65d-9b7817640da2 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:10.009811 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04d6a6c-4d6b-405b-97f2-8ebbb7b3ca3e admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:10.041503 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca92de88-29e9-440e-9d98-e620300b171d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:10.047258 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca92de88-29e9-440e-9d98-e620300b171d admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:10.049279 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca92de88-29e9-440e-9d98-e620300b171d admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:10.049663 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca92de88-29e9-440e-9d98-e620300b171d admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:10.050444 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca92de88-29e9-440e-9d98-e620300b171d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:10.061644 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 843/1684] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:14:09 2022] PUT /identity/v3/projects/dab58d1c4b6b48d88067f316d5aa0120/users/e81659c46f6d4354a0897e4fde79d1a2/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:10.061644 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664b2d04-05e0-4b9e-a65d-9b7817640da2 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:10.064934 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-664b2d04-05e0-4b9e-a65d-9b7817640da2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:10.072753 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca92de88-29e9-440e-9d98-e620300b171d admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:10.135163 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 842/1685] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:14:09 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:14:10.143593 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6d7b6e-5d1e-44f2-8a39-6133fe1fac43 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:10.143894 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6d7b6e-5d1e-44f2-8a39-6133fe1fac43 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:10.144174 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6d7b6e-5d1e-44f2-8a39-6133fe1fac43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:10.273658 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:10.285035 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-9d6d7b6e-5d1e-44f2-8a39-6133fe1fac43 None None] MFA Rules not processed for user `e81659c46f6d4354a0897e4fde79d1a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:10.301038 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-9d6d7b6e-5d1e-44f2-8a39-6133fe1fac43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:10.336921 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664b2d04-05e0-4b9e-a65d-9b7817640da2 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:10.361275 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff58ac11-fb32-4d91-877a-b52ad008bee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:10.361936 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff58ac11-fb32-4d91-877a-b52ad008bee6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:10.362221 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff58ac11-fb32-4d91-877a-b52ad008bee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:10.362500 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff58ac11-fb32-4d91-877a-b52ad008bee6 admin admin] PATH_INFO: `/v3/projects/e51d6377f5134fb0baf62d1cf6184658` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:10.363885 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff58ac11-fb32-4d91-877a-b52ad008bee6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e51d6377f5134fb0baf62d1cf6184658)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:10.365855 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff58ac11-fb32-4d91-877a-b52ad008bee6 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:10.484302 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-ff58ac11-fb32-4d91-877a-b52ad008bee6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e51d6377f5134fb0baf62d1cf6184658', 'request_id': 'req-ff58ac11-fb32-4d91-877a-b52ad008bee6'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:10.486410 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 843/1686] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:10 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:10.487259 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6d7b6e-5d1e-44f2-8a39-6133fe1fac43 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:10.499964 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-ff58ac11-fb32-4d91-877a-b52ad008bee6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e51d6377f5134fb0baf62d1cf6184658', 'request_id': 'req-ff58ac11-fb32-4d91-877a-b52ad008bee6'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:10.508398 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6d7b6e-5d1e-44f2-8a39-6133fe1fac43 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:10.513698 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-ff58ac11-fb32-4d91-877a-b52ad008bee6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e51d6377f5134fb0baf62d1cf6184658', 'request_id': 'req-ff58ac11-fb32-4d91-877a-b52ad008bee6'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:10.530444 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-ff58ac11-fb32-4d91-877a-b52ad008bee6 admin admin] The token cache is being invalidate because project e51d6377f5134fb0baf62d1cf6184658 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81712) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:14:10.530764 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-ff58ac11-fb32-4d91-877a-b52ad008bee6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:10.532261 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-ff58ac11-fb32-4d91-877a-b52ad008bee6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:10.539314 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 844/1687] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:14:10 2022] DELETE /identity/v3/projects/e51d6377f5134fb0baf62d1cf6184658 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:10.541551 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff58ac11-fb32-4d91-877a-b52ad008bee6 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:10.545076 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-ff58ac11-fb32-4d91-877a-b52ad008bee6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:10.704033 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c5818c-111a-4966-a702-885ad45708b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:10.704728 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c5818c-111a-4966-a702-885ad45708b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:10.705004 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c5818c-111a-4966-a702-885ad45708b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:10.705285 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c5818c-111a-4966-a702-885ad45708b4 admin admin] PATH_INFO: `/v3/projects/dbd0751749af4d16ad697394257ba9c5/users/64ea2f60612b44e1b2c0c7b48ff61517/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:10.749315 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff58ac11-fb32-4d91-877a-b52ad008bee6 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:10.759374 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c5818c-111a-4966-a702-885ad45708b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbd0751749af4d16ad697394257ba9c5, user_id=64ea2f60612b44e1b2c0c7b48ff61517, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:10.765382 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c5818c-111a-4966-a702-885ad45708b4 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:10.767867 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e96822-e382-4273-ad7d-3a497d4988e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:10.768492 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e96822-e382-4273-ad7d-3a497d4988e0 service cinder] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:10.770047 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e96822-e382-4273-ad7d-3a497d4988e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:10.770466 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e96822-e382-4273-ad7d-3a497d4988e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:10.777557 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-b3e96822-e382-4273-ad7d-3a497d4988e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:10.811265 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 844/1688] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:14:10 2022] PUT /identity/v3/projects/dbd0751749af4d16ad697394257ba9c5/users/64ea2f60612b44e1b2c0c7b48ff61517/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:10.811265 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bc2e72-f4a8-4a98-95da-ea49be754cc5 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:10.811265 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bc2e72-f4a8-4a98-95da-ea49be754cc5 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:10.811265 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bc2e72-f4a8-4a98-95da-ea49be754cc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:10.859546 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-05bc2e72-f4a8-4a98-95da-ea49be754cc5 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:10.862279 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-05bc2e72-f4a8-4a98-95da-ea49be754cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:10.883145 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:10.947132 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e96822-e382-4273-ad7d-3a497d4988e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:10.951923 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e96822-e382-4273-ad7d-3a497d4988e0 service cinder] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:10.974919 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 845/1689] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:14:10 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:14:10.976814 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e96822-e382-4273-ad7d-3a497d4988e0 service cinder] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:10.980077 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-b3e96822-e382-4273-ad7d-3a497d4988e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:11.038103 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e96822-e382-4273-ad7d-3a497d4988e0 service cinder] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:11.042320 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 845/1690] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:11.049986 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338aa32-4055-4fd4-869f-5fe21863b80c None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:11.050473 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338aa32-4055-4fd4-869f-5fe21863b80c None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:11.050932 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338aa32-4055-4fd4-869f-5fe21863b80c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:11.052265 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ee31f7-c7fc-45db-b4ac-4a24d0e4e45c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:11.066846 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee31f7-c7fc-45db-b4ac-4a24d0e4e45c admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:11.067708 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee31f7-c7fc-45db-b4ac-4a24d0e4e45c admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:11.068136 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee31f7-c7fc-45db-b4ac-4a24d0e4e45c admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:11.073149 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ee31f7-c7fc-45db-b4ac-4a24d0e4e45c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:11.078746 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ee31f7-c7fc-45db-b4ac-4a24d0e4e45c admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:11.117564 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 846/1691] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:14:10 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:14:11.117564 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ee31f7-c7fc-45db-b4ac-4a24d0e4e45c admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:11.123085 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-37ee31f7-c7fc-45db-b4ac-4a24d0e4e45c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:11.124111 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-6338aa32-4055-4fd4-869f-5fe21863b80c None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:11.126959 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-6338aa32-4055-4fd4-869f-5fe21863b80c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:11.150480 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 846/1692] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:11.150480 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6338aa32-4055-4fd4-869f-5fe21863b80c None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:11.154282 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ee31f7-c7fc-45db-b4ac-4a24d0e4e45c admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:11.163903 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9496198-059f-42d7-a165-3d17e7b833cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:11.164556 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9496198-059f-42d7-a165-3d17e7b833cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:11.164839 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9496198-059f-42d7-a165-3d17e7b833cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:11.166839 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9496198-059f-42d7-a165-3d17e7b833cb admin admin] PATH_INFO: `/v3/users/a00230e9e90448a5ad1261ccd67e8d2f` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:11.168335 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9496198-059f-42d7-a165-3d17e7b833cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=a00230e9e90448a5ad1261ccd67e8d2f)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:11.171804 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6338aa32-4055-4fd4-869f-5fe21863b80c None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:11.174092 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9496198-059f-42d7-a165-3d17e7b833cb admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:11.180371 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcaaad2-071f-46e9-aad5-251ed409aa1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:11.182386 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcaaad2-071f-46e9-aad5-251ed409aa1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:11.182386 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcaaad2-071f-46e9-aad5-251ed409aa1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:11.184792 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:11.184934 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcaaad2-071f-46e9-aad5-251ed409aa1f admin admin] PATH_INFO: `/v3/projects/dbd0751749af4d16ad697394257ba9c5/users/64ea2f60612b44e1b2c0c7b48ff61517/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:11.192530 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcaaad2-071f-46e9-aad5-251ed409aa1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbd0751749af4d16ad697394257ba9c5, user_id=64ea2f60612b44e1b2c0c7b48ff61517, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:11.203801 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcaaad2-071f-46e9-aad5-251ed409aa1f admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:11.255517 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 847/1693] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:14:11 2022] PUT /identity/v3/projects/dbd0751749af4d16ad697394257ba9c5/users/64ea2f60612b44e1b2c0c7b48ff61517/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:11.255517 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcaaad2-071f-46e9-aad5-251ed409aa1f admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:11.310747 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-d9496198-059f-42d7-a165-3d17e7b833cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a00230e9e90448a5ad1261ccd67e8d2f'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:11.332434 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-d9496198-059f-42d7-a165-3d17e7b833cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a00230e9e90448a5ad1261ccd67e8d2f'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:11.333204 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-d9496198-059f-42d7-a165-3d17e7b833cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a00230e9e90448a5ad1261ccd67e8d2f'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:11.343584 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-d9496198-059f-42d7-a165-3d17e7b833cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a00230e9e90448a5ad1261ccd67e8d2f'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:11.346405 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-d9496198-059f-42d7-a165-3d17e7b833cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a00230e9e90448a5ad1261ccd67e8d2f'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:11.396974 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 847/1694] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:14:11 2022] DELETE /identity/v3/users/a00230e9e90448a5ad1261ccd67e8d2f => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:11.400690 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a916114-5985-44ca-878e-2d6c95974d4f None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:11.400690 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a916114-5985-44ca-878e-2d6c95974d4f None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:11.400690 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a916114-5985-44ca-878e-2d6c95974d4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:11.438438 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcaaad2-071f-46e9-aad5-251ed409aa1f admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:11.486761 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-8a916114-5985-44ca-878e-2d6c95974d4f None None] MFA Rules not processed for user `64ea2f60612b44e1b2c0c7b48ff61517`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:11.489924 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-8a916114-5985-44ca-878e-2d6c95974d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:11.625617 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fecdd3-8619-4105-8dbe-c80661302072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:11.628573 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fecdd3-8619-4105-8dbe-c80661302072 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:11.629043 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fecdd3-8619-4105-8dbe-c80661302072 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:11.643312 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fecdd3-8619-4105-8dbe-c80661302072 admin admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:11.644747 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fecdd3-8619-4105-8dbe-c80661302072 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:14:11.647505 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fecdd3-8619-4105-8dbe-c80661302072 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:11.653073 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fecdd3-8619-4105-8dbe-c80661302072 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:11.697611 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 848/1695] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:14:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 451 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:14:11.698950 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92513eba-c420-4188-9eda-d19c1b913652 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:11.699258 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92513eba-c420-4188-9eda-d19c1b913652 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:11.699548 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92513eba-c420-4188-9eda-d19c1b913652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:11.734737 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 848/1696] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:11 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:11.743813 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a916114-5985-44ca-878e-2d6c95974d4f None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:11.746643 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-8a916114-5985-44ca-878e-2d6c95974d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:11.781433 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-92513eba-c420-4188-9eda-d19c1b913652 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:11.782713 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-92513eba-c420-4188-9eda-d19c1b913652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:11.784756 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a916114-5985-44ca-878e-2d6c95974d4f None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:11.799265 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2e5427-886d-43e2-ac8b-5ad4ece5a9c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:11.799908 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2e5427-886d-43e2-ac8b-5ad4ece5a9c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:11.800195 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2e5427-886d-43e2-ac8b-5ad4ece5a9c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:11.800472 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2e5427-886d-43e2-ac8b-5ad4ece5a9c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:11.805977 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2e5427-886d-43e2-ac8b-5ad4ece5a9c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:11.806436 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 849/1697] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:11.808073 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92513eba-c420-4188-9eda-d19c1b913652 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:11.811589 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2e5427-886d-43e2-ac8b-5ad4ece5a9c1 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:11.820589 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-92513eba-c420-4188-9eda-d19c1b913652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:11.885642 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:11.888584 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 849/1698] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:14:11 2022] POST /identity/v3/projects => generated 373 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:14:11.891657 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2e5427-886d-43e2-ac8b-5ad4ece5a9c1 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:11.955149 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92513eba-c420-4188-9eda-d19c1b913652 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:12.049301 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:12.081122 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c551942-3eda-42f5-9ca9-f9b9f3b202a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:12.081754 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c551942-3eda-42f5-9ca9-f9b9f3b202a3 service cinder] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:12.082176 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c551942-3eda-42f5-9ca9-f9b9f3b202a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:12.082450 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c551942-3eda-42f5-9ca9-f9b9f3b202a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:12.094404 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2e5427-886d-43e2-ac8b-5ad4ece5a9c1 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:12.107081 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ebd51-4efd-4395-be24-a07fc8ce47d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:12.107665 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ebd51-4efd-4395-be24-a07fc8ce47d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:12.107950 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ebd51-4efd-4395-be24-a07fc8ce47d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:12.109965 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ebd51-4efd-4395-be24-a07fc8ce47d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:12.110647 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085ebd51-4efd-4395-be24-a07fc8ce47d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:12.117108 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085ebd51-4efd-4395-be24-a07fc8ce47d8 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:12.169442 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 850/1699] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:14:11 2022] POST /identity/v3/users => generated 287 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:14:12.170866 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ebd51-4efd-4395-be24-a07fc8ce47d8 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:12.175766 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-085ebd51-4efd-4395-be24-a07fc8ce47d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:12.234250 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c551942-3eda-42f5-9ca9-f9b9f3b202a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:12.235742 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c551942-3eda-42f5-9ca9-f9b9f3b202a3 service cinder] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:12.266285 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 850/1700] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:14:11 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:14:12.267170 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c551942-3eda-42f5-9ca9-f9b9f3b202a3 service cinder] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:12.279177 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c551942-3eda-42f5-9ca9-f9b9f3b202a3 service cinder] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:12.286254 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ba6ae8-bfda-428f-b984-56a22ed7b6c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:12.286958 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba6ae8-bfda-428f-b984-56a22ed7b6c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:12.287268 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba6ae8-bfda-428f-b984-56a22ed7b6c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:12.287544 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba6ae8-bfda-428f-b984-56a22ed7b6c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:12.288322 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ba6ae8-bfda-428f-b984-56a22ed7b6c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:12.290289 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ba6ae8-bfda-428f-b984-56a22ed7b6c6 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:12.306183 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 851/1701] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:14:12.308055 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ba6ae8-bfda-428f-b984-56a22ed7b6c6 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:12.319372 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ba6ae8-bfda-428f-b984-56a22ed7b6c6 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:12.327474 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b279857-539f-476f-8990-e6bba80fe4fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:12.328813 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b279857-539f-476f-8990-e6bba80fe4fc admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:12.328813 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b279857-539f-476f-8990-e6bba80fe4fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:12.328813 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b279857-539f-476f-8990-e6bba80fe4fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:12.329266 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b279857-539f-476f-8990-e6bba80fe4fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:12.331095 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b279857-539f-476f-8990-e6bba80fe4fc admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:12.342939 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 852/1702] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:14:12.344312 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ebd51-4efd-4395-be24-a07fc8ce47d8 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:12.348799 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b279857-539f-476f-8990-e6bba80fe4fc admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:12.356216 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e2fb5e-9ebe-4ab1-b9a8-c3893348a0cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:12.356864 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e2fb5e-9ebe-4ab1-b9a8-c3893348a0cd service neutron] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:12.357203 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e2fb5e-9ebe-4ab1-b9a8-c3893348a0cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:12.357418 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e2fb5e-9ebe-4ab1-b9a8-c3893348a0cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:12.360099 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b279857-539f-476f-8990-e6bba80fe4fc admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:12.365354 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55838ad4-d79f-48e4-8090-17c6c2d350e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:12.366061 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55838ad4-d79f-48e4-8090-17c6c2d350e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:12.366345 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55838ad4-d79f-48e4-8090-17c6c2d350e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:12.366849 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55838ad4-d79f-48e4-8090-17c6c2d350e5 admin admin] PATH_INFO: `/v3/projects/78ba2c038ceb46b9b5bbf241b2a55a1a/users/1d248594dc5a49dfb10c074d34947762/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:12.383025 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e2fb5e-9ebe-4ab1-b9a8-c3893348a0cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:12.384978 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55838ad4-d79f-48e4-8090-17c6c2d350e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78ba2c038ceb46b9b5bbf241b2a55a1a, user_id=1d248594dc5a49dfb10c074d34947762, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:12.388864 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55838ad4-d79f-48e4-8090-17c6c2d350e5 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:12.389181 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e2fb5e-9ebe-4ab1-b9a8-c3893348a0cd service neutron] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:12.418485 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 851/1703] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:14:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:14:12.422021 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 853/1704] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:14:12 2022] PUT /identity/v3/projects/78ba2c038ceb46b9b5bbf241b2a55a1a/users/1d248594dc5a49dfb10c074d34947762/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:12.422145 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e2fb5e-9ebe-4ab1-b9a8-c3893348a0cd service neutron] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:12.423888 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-f6e2fb5e-9ebe-4ab1-b9a8-c3893348a0cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:12.435151 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabcacb6-fa11-4060-aa3d-96ef747de6b0 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:12.435476 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabcacb6-fa11-4060-aa3d-96ef747de6b0 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:12.435738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabcacb6-fa11-4060-aa3d-96ef747de6b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:12.456343 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e2fb5e-9ebe-4ab1-b9a8-c3893348a0cd service neutron] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:12.518270 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-aabcacb6-fa11-4060-aa3d-96ef747de6b0 None None] MFA Rules not processed for user `1d248594dc5a49dfb10c074d34947762`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:12.520117 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-aabcacb6-fa11-4060-aa3d-96ef747de6b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:12.625867 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:12.637086 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dc8d2e-193c-4838-9a98-1971ea519a7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:12.637758 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc8d2e-193c-4838-9a98-1971ea519a7d service cinder] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:12.638045 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc8d2e-193c-4838-9a98-1971ea519a7d service cinder] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:12.638325 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc8d2e-193c-4838-9a98-1971ea519a7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:12.648303 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-80dc8d2e-193c-4838-9a98-1971ea519a7d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:12.692785 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 854/1705] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:12.698989 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabcacb6-fa11-4060-aa3d-96ef747de6b0 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:12.823596 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dc8d2e-193c-4838-9a98-1971ea519a7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:12.828137 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dc8d2e-193c-4838-9a98-1971ea519a7d service cinder] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:12.843850 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabcacb6-fa11-4060-aa3d-96ef747de6b0 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:12.860877 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36bc313-98c2-455d-8fbc-325a7d431bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:12.861506 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36bc313-98c2-455d-8fbc-325a7d431bb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:12.861786 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36bc313-98c2-455d-8fbc-325a7d431bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:12.862060 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36bc313-98c2-455d-8fbc-325a7d431bb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:12.862391 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 852/1706] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:14:12 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:14:12.862700 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36bc313-98c2-455d-8fbc-325a7d431bb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:12.864702 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36bc313-98c2-455d-8fbc-325a7d431bb8 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:12.940936 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 855/1707] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:14:12 2022] POST /identity/v3/projects => generated 371 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:14:12.944960 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dc8d2e-193c-4838-9a98-1971ea519a7d service cinder] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:12.969506 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dc8d2e-193c-4838-9a98-1971ea519a7d service cinder] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:13.055652 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:13.156231 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6876c91c-cc5d-4c30-b0eb-2a093237a424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:13.158855 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6876c91c-cc5d-4c30-b0eb-2a093237a424 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:13.160461 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6876c91c-cc5d-4c30-b0eb-2a093237a424 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:13.161260 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6876c91c-cc5d-4c30-b0eb-2a093237a424 admin admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:13.163851 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6876c91c-cc5d-4c30-b0eb-2a093237a424 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:13.170923 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6876c91c-cc5d-4c30-b0eb-2a093237a424 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:13.253559 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 853/1708] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:14:12 2022] POST /identity/v3/users => generated 292 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:14:13.258470 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36bc313-98c2-455d-8fbc-325a7d431bb8 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:13.273909 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36bc313-98c2-455d-8fbc-325a7d431bb8 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:13.290304 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464e7689-c3cb-4082-998d-609404d7328c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:13.294725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464e7689-c3cb-4082-998d-609404d7328c admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:13.295235 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464e7689-c3cb-4082-998d-609404d7328c admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:13.295528 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464e7689-c3cb-4082-998d-609404d7328c admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:13.297903 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464e7689-c3cb-4082-998d-609404d7328c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:13.302117 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464e7689-c3cb-4082-998d-609404d7328c admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:13.333807 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 856/1709] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:14:13 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:14:13.338670 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6876c91c-cc5d-4c30-b0eb-2a093237a424 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:13.369492 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6876c91c-cc5d-4c30-b0eb-2a093237a424 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:13.388122 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e22384-3d20-4359-9aeb-d9ad434f12af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:13.392747 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e22384-3d20-4359-9aeb-d9ad434f12af admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:13.394571 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e22384-3d20-4359-9aeb-d9ad434f12af admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:13.394571 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e22384-3d20-4359-9aeb-d9ad434f12af admin admin] PATH_INFO: `/v3/projects/d8522abdf0774a7993fc55c493ccfc8d/users/40ed2a2a4ea34a73b778b979f7c06b66/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:13.453513 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e22384-3d20-4359-9aeb-d9ad434f12af admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8522abdf0774a7993fc55c493ccfc8d, user_id=40ed2a2a4ea34a73b778b979f7c06b66, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:13.465120 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e22384-3d20-4359-9aeb-d9ad434f12af admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:13.505533 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 854/1710] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:14:13 2022] PUT /identity/v3/projects/d8522abdf0774a7993fc55c493ccfc8d/users/40ed2a2a4ea34a73b778b979f7c06b66/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 12:14:13.511771 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464e7689-c3cb-4082-998d-609404d7328c admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:13.527681 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464e7689-c3cb-4082-998d-609404d7328c admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:13.655792 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89ada57-6b17-4661-a72d-c56b17eb663d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:13.656866 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ada57-6b17-4661-a72d-c56b17eb663d admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:13.657397 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ada57-6b17-4661-a72d-c56b17eb663d admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:13.657845 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ada57-6b17-4661-a72d-c56b17eb663d admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:13.658639 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89ada57-6b17-4661-a72d-c56b17eb663d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:13.662448 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89ada57-6b17-4661-a72d-c56b17eb663d admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:13.678721 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 857/1711] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:14:13 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:14:13.683177 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e22384-3d20-4359-9aeb-d9ad434f12af admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:13.708328 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e22384-3d20-4359-9aeb-d9ad434f12af admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:13.716726 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f1db57-483d-40e2-a80e-d3889e33e7a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:13.717320 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f1db57-483d-40e2-a80e-d3889e33e7a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:13.717639 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f1db57-483d-40e2-a80e-d3889e33e7a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:13.720060 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f1db57-483d-40e2-a80e-d3889e33e7a1 admin admin] PATH_INFO: `/v3/projects/d8522abdf0774a7993fc55c493ccfc8d/users/40ed2a2a4ea34a73b778b979f7c06b66/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:13.725367 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f1db57-483d-40e2-a80e-d3889e33e7a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8522abdf0774a7993fc55c493ccfc8d, user_id=40ed2a2a4ea34a73b778b979f7c06b66, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:13.732645 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f1db57-483d-40e2-a80e-d3889e33e7a1 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:13.762154 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 855/1712] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:14:13 2022] PUT /identity/v3/projects/d8522abdf0774a7993fc55c493ccfc8d/users/40ed2a2a4ea34a73b778b979f7c06b66/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:13.774489 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbde47a-8cc3-4cc8-af7e-c8da3e0fed54 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:13.775022 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbde47a-8cc3-4cc8-af7e-c8da3e0fed54 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:13.775743 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbde47a-8cc3-4cc8-af7e-c8da3e0fed54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:13.814727 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-7cbde47a-8cc3-4cc8-af7e-c8da3e0fed54 None None] MFA Rules not processed for user `40ed2a2a4ea34a73b778b979f7c06b66`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:13.816151 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-7cbde47a-8cc3-4cc8-af7e-c8da3e0fed54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:13.890040 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:13.950785 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 858/1713] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:13 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:13.964094 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f1db57-483d-40e2-a80e-d3889e33e7a1 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:14.072657 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f1db57-483d-40e2-a80e-d3889e33e7a1 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:14.078647 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe303ea-6f73-43cb-8197-a4290794822a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:14.079606 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe303ea-6f73-43cb-8197-a4290794822a service cinder] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:14.079963 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe303ea-6f73-43cb-8197-a4290794822a service cinder] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:14.080617 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe303ea-6f73-43cb-8197-a4290794822a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:14.121061 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe303ea-6f73-43cb-8197-a4290794822a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:14.125244 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6208730-c07d-4399-992a-81f9daf0c35b None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:14.125244 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6208730-c07d-4399-992a-81f9daf0c35b None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:14.129471 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6208730-c07d-4399-992a-81f9daf0c35b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:14.141674 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe303ea-6f73-43cb-8197-a4290794822a service cinder] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:14.203832 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 856/1714] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:14:13 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:14:14.237710 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-d6208730-c07d-4399-992a-81f9daf0c35b None None] MFA Rules not processed for user `a02508c20bd547708638c1875101da6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:14.239049 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-d6208730-c07d-4399-992a-81f9daf0c35b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:14.422762 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe303ea-6f73-43cb-8197-a4290794822a service cinder] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:14.438066 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 859/1715] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:14 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:14:14.440618 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe303ea-6f73-43cb-8197-a4290794822a service cinder] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:14.450667 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb5f42-23c0-48ab-89b8-d1320ea18f62 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:14.450959 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb5f42-23c0-48ab-89b8-d1320ea18f62 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:14.451342 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb5f42-23c0-48ab-89b8-d1320ea18f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:14.452033 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731f0ddc-1e34-45f9-bb45-98589a79d685 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:14.452855 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f0ddc-1e34-45f9-bb45-98589a79d685 service cinder] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:14.452978 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f0ddc-1e34-45f9-bb45-98589a79d685 service cinder] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:14.456345 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f0ddc-1e34-45f9-bb45-98589a79d685 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:14.481674 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-b6bb5f42-23c0-48ab-89b8-d1320ea18f62 None None] MFA Rules not processed for user `a02508c20bd547708638c1875101da6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:14.482595 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-b6bb5f42-23c0-48ab-89b8-d1320ea18f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:14.492667 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 860/1716] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:14 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:14.496530 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bb5f42-23c0-48ab-89b8-d1320ea18f62 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:14.498251 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-b6bb5f42-23c0-48ab-89b8-d1320ea18f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:14.625831 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731f0ddc-1e34-45f9-bb45-98589a79d685 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:14.627628 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731f0ddc-1e34-45f9-bb45-98589a79d685 service cinder] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:14.671718 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 857/1717] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:14:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:14:14.675498 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731f0ddc-1e34-45f9-bb45-98589a79d685 service cinder] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:14.683153 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-731f0ddc-1e34-45f9-bb45-98589a79d685 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:14.765125 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bb5f42-23c0-48ab-89b8-d1320ea18f62 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:14.769804 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dad3e2c-058e-4616-98b5-4bac4ad4acc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:14.770462 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad3e2c-058e-4616-98b5-4bac4ad4acc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:14.770742 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad3e2c-058e-4616-98b5-4bac4ad4acc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:14.771021 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad3e2c-058e-4616-98b5-4bac4ad4acc1 admin admin] PATH_INFO: `/v3/projects/fcfa7283ecf84b41ac3725ea099b4835` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:14.778407 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dad3e2c-058e-4616-98b5-4bac4ad4acc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcfa7283ecf84b41ac3725ea099b4835)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:14.779703 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: WARNING py.warnings [None req-3dad3e2c-058e-4616-98b5-4bac4ad4acc1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 26 12:14:14.779703 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: warnings.warn(msg) Aug 26 12:14:14.779703 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: Aug 26 12:14:14.781607 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dad3e2c-058e-4616-98b5-4bac4ad4acc1 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:14.824139 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:14.944421 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-3dad3e2c-058e-4616-98b5-4bac4ad4acc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcfa7283ecf84b41ac3725ea099b4835', 'request_id': 'req-3dad3e2c-058e-4616-98b5-4bac4ad4acc1'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:15.012715 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-3dad3e2c-058e-4616-98b5-4bac4ad4acc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcfa7283ecf84b41ac3725ea099b4835', 'request_id': 'req-3dad3e2c-058e-4616-98b5-4bac4ad4acc1'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:15.028385 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:15.044596 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-3dad3e2c-058e-4616-98b5-4bac4ad4acc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcfa7283ecf84b41ac3725ea099b4835', 'request_id': 'req-3dad3e2c-058e-4616-98b5-4bac4ad4acc1'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:15.078102 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731f0ddc-1e34-45f9-bb45-98589a79d685 service cinder] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:15.084816 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-3dad3e2c-058e-4616-98b5-4bac4ad4acc1 admin admin] The token cache is being invalidate because project fcfa7283ecf84b41ac3725ea099b4835 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:14:15.085129 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-3dad3e2c-058e-4616-98b5-4bac4ad4acc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:15.091949 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-3dad3e2c-058e-4616-98b5-4bac4ad4acc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:15.099367 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 861/1718] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:14:14 2022] DELETE /identity/v3/projects/fcfa7283ecf84b41ac3725ea099b4835 => generated 0 bytes in 603 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:15.101755 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dad3e2c-058e-4616-98b5-4bac4ad4acc1 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:15.108200 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-3dad3e2c-058e-4616-98b5-4bac4ad4acc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:15.253995 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9681055a-ad0a-4883-bbb9-874b7af6b395 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aacd7d6d17e4f10ba5c9107a12beafa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:15.255045 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9681055a-ad0a-4883-bbb9-874b7af6b395 service nova] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:15.255566 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9681055a-ad0a-4883-bbb9-874b7af6b395 service nova] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:15.256092 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9681055a-ad0a-4883-bbb9-874b7af6b395 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:15.260445 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-9681055a-ad0a-4883-bbb9-874b7af6b395 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:15.311492 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dad3e2c-058e-4616-98b5-4bac4ad4acc1 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:15.323184 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de8e3d0-1e3e-407f-8342-bc8a5ae06864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:15.323875 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de8e3d0-1e3e-407f-8342-bc8a5ae06864 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:15.324397 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de8e3d0-1e3e-407f-8342-bc8a5ae06864 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:15.324397 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de8e3d0-1e3e-407f-8342-bc8a5ae06864 admin admin] PATH_INFO: `/v3/users/dca46c79bb614edb9c544031bbfad47e` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:15.327714 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de8e3d0-1e3e-407f-8342-bc8a5ae06864 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dca46c79bb614edb9c544031bbfad47e)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:15.333783 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de8e3d0-1e3e-407f-8342-bc8a5ae06864 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:15.429792 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9681055a-ad0a-4883-bbb9-874b7af6b395 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:15.441094 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9681055a-ad0a-4883-bbb9-874b7af6b395 service nova] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:15.458067 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 858/1719] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:14:14 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 785 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:14:15.459634 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9681055a-ad0a-4883-bbb9-874b7af6b395 service nova] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:15.461425 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-6de8e3d0-1e3e-407f-8342-bc8a5ae06864 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dca46c79bb614edb9c544031bbfad47e'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:15.463838 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-9681055a-ad0a-4883-bbb9-874b7af6b395 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:15.483273 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9681055a-ad0a-4883-bbb9-874b7af6b395 service nova] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:15.496081 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-6de8e3d0-1e3e-407f-8342-bc8a5ae06864 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dca46c79bb614edb9c544031bbfad47e'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:15.498438 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa3cd4-ad08-461c-96fa-329e164aa837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:15.500705 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa3cd4-ad08-461c-96fa-329e164aa837 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:15.501219 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa3cd4-ad08-461c-96fa-329e164aa837 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:15.501851 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa3cd4-ad08-461c-96fa-329e164aa837 admin admin] PATH_INFO: `/v3/users/e81659c46f6d4354a0897e4fde79d1a2` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:15.504308 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aa3cd4-ad08-461c-96fa-329e164aa837 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e81659c46f6d4354a0897e4fde79d1a2)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:15.509146 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aa3cd4-ad08-461c-96fa-329e164aa837 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:15.511603 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-6de8e3d0-1e3e-407f-8342-bc8a5ae06864 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dca46c79bb614edb9c544031bbfad47e'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:15.512578 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-6de8e3d0-1e3e-407f-8342-bc8a5ae06864 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dca46c79bb614edb9c544031bbfad47e'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:15.537625 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-6de8e3d0-1e3e-407f-8342-bc8a5ae06864 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dca46c79bb614edb9c544031bbfad47e'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:15.553952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 862/1720] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:14:15 2022] DELETE /identity/v3/users/dca46c79bb614edb9c544031bbfad47e => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:15.559243 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb1cce-58d3-4ebc-ba00-81b1a9c31124 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:15.559792 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb1cce-58d3-4ebc-ba00-81b1a9c31124 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:15.560255 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb1cce-58d3-4ebc-ba00-81b1a9c31124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:15.594865 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-aeeb1cce-58d3-4ebc-ba00-81b1a9c31124 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:15.596497 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-aeeb1cce-58d3-4ebc-ba00-81b1a9c31124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:15.643239 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-29aa3cd4-ad08-461c-96fa-329e164aa837 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e81659c46f6d4354a0897e4fde79d1a2'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:15.670008 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-29aa3cd4-ad08-461c-96fa-329e164aa837 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e81659c46f6d4354a0897e4fde79d1a2'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:15.670990 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-29aa3cd4-ad08-461c-96fa-329e164aa837 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e81659c46f6d4354a0897e4fde79d1a2'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:15.673137 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:15.693245 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-29aa3cd4-ad08-461c-96fa-329e164aa837 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e81659c46f6d4354a0897e4fde79d1a2'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:15.694239 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-29aa3cd4-ad08-461c-96fa-329e164aa837 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e81659c46f6d4354a0897e4fde79d1a2'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:15.854088 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 859/1721] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:14:15 2022] DELETE /identity/v3/users/e81659c46f6d4354a0897e4fde79d1a2 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:16.123821 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 863/1722] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 565 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:16.157886 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa3cd4-ad08-461c-96fa-329e164aa837 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:16.168713 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-29aa3cd4-ad08-461c-96fa-329e164aa837 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:16.459939 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa3cd4-ad08-461c-96fa-329e164aa837 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:16.473665 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f65c37-0154-49d3-bfc6-877a81573df2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:16.475132 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f65c37-0154-49d3-bfc6-877a81573df2 service neutron] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:16.475452 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f65c37-0154-49d3-bfc6-877a81573df2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:16.475734 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f65c37-0154-49d3-bfc6-877a81573df2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:16.668744 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f65c37-0154-49d3-bfc6-877a81573df2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:16.670215 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f65c37-0154-49d3-bfc6-877a81573df2 service neutron] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:16.704886 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 860/1723] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:14:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 546 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:14:17.252668 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c3faa6-bb98-4758-9195-9ecda2519a36 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:17.253487 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c3faa6-bb98-4758-9195-9ecda2519a36 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:17.254370 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c3faa6-bb98-4758-9195-9ecda2519a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:17.275753 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:17.314052 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-22c3faa6-bb98-4758-9195-9ecda2519a36 None None] MFA Rules not processed for user `2d9f1a5e4e77467f8a691b634a841de2`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:17.315482 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-22c3faa6-bb98-4758-9195-9ecda2519a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:17.452694 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 864/1724] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:17.469657 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f65c37-0154-49d3-bfc6-877a81573df2 service neutron] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:17.471144 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-86f65c37-0154-49d3-bfc6-877a81573df2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:17.601808 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1726f7c7-a297-45ed-8b1d-46e9883a2e23 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:17.602769 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1726f7c7-a297-45ed-8b1d-46e9883a2e23 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:17.603274 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1726f7c7-a297-45ed-8b1d-46e9883a2e23 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:17.605395 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 865/1725] 10.222.0.61 () {54 vars in 912 bytes} [Fri Aug 26 12:14:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 12:14:17.613708 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d5bbd0-0ddb-4b80-847e-b73a6b063bd5 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:17.614207 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d5bbd0-0ddb-4b80-847e-b73a6b063bd5 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:17.614902 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d5bbd0-0ddb-4b80-847e-b73a6b063bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:17.680169 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f65c37-0154-49d3-bfc6-877a81573df2 service neutron] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:17.692662 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc16ed-e9e5-405b-b33e-ac592b95c440 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aacd7d6d17e4f10ba5c9107a12beafa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:17.693286 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc16ed-e9e5-405b-b33e-ac592b95c440 service nova] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:17.693562 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc16ed-e9e5-405b-b33e-ac592b95c440 service nova] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:17.693833 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc16ed-e9e5-405b-b33e-ac592b95c440 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:17.718002 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdc16ed-e9e5-405b-b33e-ac592b95c440 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:17.719765 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdc16ed-e9e5-405b-b33e-ac592b95c440 service nova] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:17.725108 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-f1d5bbd0-0ddb-4b80-847e-b73a6b063bd5 None None] MFA Rules not processed for user `b0972dbd5cff4610b239326425e6351c`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:17.731228 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-f1d5bbd0-0ddb-4b80-847e-b73a6b063bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:17.733565 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 861/1726] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:14:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:14:17.764121 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 866/1727] 10.222.0.61 () {58 vars in 1009 bytes} [Fri Aug 26 12:14:17 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:14:17.934880 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc16ed-e9e5-405b-b33e-ac592b95c440 service nova] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:17.956384 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc16ed-e9e5-405b-b33e-ac592b95c440 service nova] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:17.973882 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeafb50-8dda-4fe7-b64a-ece30274c83d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:17.974813 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeafb50-8dda-4fe7-b64a-ece30274c83d service neutron] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:17.975278 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeafb50-8dda-4fe7-b64a-ece30274c83d service neutron] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:17.975694 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeafb50-8dda-4fe7-b64a-ece30274c83d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:18.006971 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:18.038436 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeafb50-8dda-4fe7-b64a-ece30274c83d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:18.046249 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeafb50-8dda-4fe7-b64a-ece30274c83d service neutron] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:18.087378 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 862/1728] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:14:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:14:18.733364 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d5bbd0-0ddb-4b80-847e-b73a6b063bd5 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:18.739838 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-f1d5bbd0-0ddb-4b80-847e-b73a6b063bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:18.769264 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d5bbd0-0ddb-4b80-847e-b73a6b063bd5 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:18.777138 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46593dbf-1b3a-4269-885f-ca1424f92dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:18.778401 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46593dbf-1b3a-4269-885f-ca1424f92dc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:18.779420 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46593dbf-1b3a-4269-885f-ca1424f92dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:18.779977 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46593dbf-1b3a-4269-885f-ca1424f92dc1 admin admin] PATH_INFO: `/v3/users/1e65951661b24f0abdae7458009becfb` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:18.782500 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46593dbf-1b3a-4269-885f-ca1424f92dc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e65951661b24f0abdae7458009becfb)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:18.785628 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46593dbf-1b3a-4269-885f-ca1424f92dc1 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:18.806731 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeafb50-8dda-4fe7-b64a-ece30274c83d service neutron] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:18.809133 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-dbeafb50-8dda-4fe7-b64a-ece30274c83d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:18.837659 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeafb50-8dda-4fe7-b64a-ece30274c83d service neutron] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:18.849995 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a89f5-c02a-4aa2-93c9-653cfd0afa44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:18.854201 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a89f5-c02a-4aa2-93c9-653cfd0afa44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:18.854850 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a89f5-c02a-4aa2-93c9-653cfd0afa44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:18.855312 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a89f5-c02a-4aa2-93c9-653cfd0afa44 admin admin] PATH_INFO: `/v3/projects/dab58d1c4b6b48d88067f316d5aa0120` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:18.857848 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5a89f5-c02a-4aa2-93c9-653cfd0afa44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dab58d1c4b6b48d88067f316d5aa0120)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:18.860377 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5a89f5-c02a-4aa2-93c9-653cfd0afa44 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:18.874601 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-46593dbf-1b3a-4269-885f-ca1424f92dc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e65951661b24f0abdae7458009becfb'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:18.887563 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-46593dbf-1b3a-4269-885f-ca1424f92dc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e65951661b24f0abdae7458009becfb'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:18.897395 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-46593dbf-1b3a-4269-885f-ca1424f92dc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e65951661b24f0abdae7458009becfb'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:18.898198 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-46593dbf-1b3a-4269-885f-ca1424f92dc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e65951661b24f0abdae7458009becfb'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:18.920853 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-46593dbf-1b3a-4269-885f-ca1424f92dc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e65951661b24f0abdae7458009becfb'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:18.932000 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 867/1729] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:14:18 2022] DELETE /identity/v3/users/1e65951661b24f0abdae7458009becfb => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:18.936512 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba1f500-a493-4e9d-9ebb-a5e913d02f8f None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:18.936949 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba1f500-a493-4e9d-9ebb-a5e913d02f8f None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:18.937367 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba1f500-a493-4e9d-9ebb-a5e913d02f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:18.946769 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-6b5a89f5-c02a-4aa2-93c9-653cfd0afa44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dab58d1c4b6b48d88067f316d5aa0120', 'request_id': 'req-6b5a89f5-c02a-4aa2-93c9-653cfd0afa44'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:18.962799 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-1ba1f500-a493-4e9d-9ebb-a5e913d02f8f None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:18.963907 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-6b5a89f5-c02a-4aa2-93c9-653cfd0afa44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dab58d1c4b6b48d88067f316d5aa0120', 'request_id': 'req-6b5a89f5-c02a-4aa2-93c9-653cfd0afa44'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:18.964804 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-1ba1f500-a493-4e9d-9ebb-a5e913d02f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:18.981979 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-6b5a89f5-c02a-4aa2-93c9-653cfd0afa44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dab58d1c4b6b48d88067f316d5aa0120', 'request_id': 'req-6b5a89f5-c02a-4aa2-93c9-653cfd0afa44'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:19.010653 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-6b5a89f5-c02a-4aa2-93c9-653cfd0afa44 admin admin] The token cache is being invalidate because project dab58d1c4b6b48d88067f316d5aa0120 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81712) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:14:19.011425 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-6b5a89f5-c02a-4aa2-93c9-653cfd0afa44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:19.012125 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-6b5a89f5-c02a-4aa2-93c9-653cfd0afa44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:19.015728 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 863/1730] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:14:18 2022] DELETE /identity/v3/projects/dab58d1c4b6b48d88067f316d5aa0120 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:19.020543 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b94670-aed8-4c46-9fca-27b9c6abf8d8 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:19.021003 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b94670-aed8-4c46-9fca-27b9c6abf8d8 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:19.021414 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b94670-aed8-4c46-9fca-27b9c6abf8d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:19.041000 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:19.064565 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 868/1731] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:14:19.072703 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba1f500-a493-4e9d-9ebb-a5e913d02f8f None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:19.074377 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-1ba1f500-a493-4e9d-9ebb-a5e913d02f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:19.084497 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-26b94670-aed8-4c46-9fca-27b9c6abf8d8 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:19.085760 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-26b94670-aed8-4c46-9fca-27b9c6abf8d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:19.098284 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 864/1732] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:19.107037 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b23daac-58fa-4dda-a397-96a71c7ef9b8 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:19.107436 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b23daac-58fa-4dda-a397-96a71c7ef9b8 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:19.107900 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b23daac-58fa-4dda-a397-96a71c7ef9b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:19.167247 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba1f500-a493-4e9d-9ebb-a5e913d02f8f None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:19.168390 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-3b23daac-58fa-4dda-a397-96a71c7ef9b8 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:19.171534 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc15ae6-6c64-424e-8b9d-72cb085ef13f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:19.172913 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-3b23daac-58fa-4dda-a397-96a71c7ef9b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:19.173190 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc15ae6-6c64-424e-8b9d-72cb085ef13f service neutron] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:19.173630 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc15ae6-6c64-424e-8b9d-72cb085ef13f service neutron] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:19.174043 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc15ae6-6c64-424e-8b9d-72cb085ef13f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:19.191482 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc15ae6-6c64-424e-8b9d-72cb085ef13f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:19.193052 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 865/1733] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:19.194862 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc15ae6-6c64-424e-8b9d-72cb085ef13f service neutron] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:19.197791 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b23daac-58fa-4dda-a397-96a71c7ef9b8 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:19.215907 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b23daac-58fa-4dda-a397-96a71c7ef9b8 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:19.219218 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 869/1734] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:14:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 12:14:19.224100 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dcf7af-f160-4bde-aa2d-1a39b8105230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:19.224906 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dcf7af-f160-4bde-aa2d-1a39b8105230 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:19.225643 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dcf7af-f160-4bde-aa2d-1a39b8105230 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:19.226064 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dcf7af-f160-4bde-aa2d-1a39b8105230 admin admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:19.227921 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dcf7af-f160-4bde-aa2d-1a39b8105230 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:14:19.228398 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dcf7af-f160-4bde-aa2d-1a39b8105230 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:19.230600 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dcf7af-f160-4bde-aa2d-1a39b8105230 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:19.250869 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 866/1735] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:14:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:14:19.252328 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc15ae6-6c64-424e-8b9d-72cb085ef13f service neutron] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:19.265707 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc15ae6-6c64-424e-8b9d-72cb085ef13f service neutron] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:19.275469 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8908f21-bc72-4dc7-8b17-baf89e9135ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:19.279395 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8908f21-bc72-4dc7-8b17-baf89e9135ef admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:19.279805 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8908f21-bc72-4dc7-8b17-baf89e9135ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:19.280703 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8908f21-bc72-4dc7-8b17-baf89e9135ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:19.282714 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8908f21-bc72-4dc7-8b17-baf89e9135ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:19.288781 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8908f21-bc72-4dc7-8b17-baf89e9135ef admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:19.364520 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 870/1736] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:14:19 2022] POST /identity/v3/projects => generated 365 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:14:19.366299 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dcf7af-f160-4bde-aa2d-1a39b8105230 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:19.406983 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dcf7af-f160-4bde-aa2d-1a39b8105230 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:19.547576 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8908f21-bc72-4dc7-8b17-baf89e9135ef admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:19.550411 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-a8908f21-bc72-4dc7-8b17-baf89e9135ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:19.596222 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336856b1-4689-4983-bb1a-1f6a09b995f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:19.597016 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336856b1-4689-4983-bb1a-1f6a09b995f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:19.597428 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336856b1-4689-4983-bb1a-1f6a09b995f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:19.597803 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336856b1-4689-4983-bb1a-1f6a09b995f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:19.598602 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336856b1-4689-4983-bb1a-1f6a09b995f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:19.601341 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336856b1-4689-4983-bb1a-1f6a09b995f3 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:19.631342 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 867/1737] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:14:19 2022] POST /identity/v3/users => generated 283 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:14:19.635444 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336856b1-4689-4983-bb1a-1f6a09b995f3 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:19.648692 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336856b1-4689-4983-bb1a-1f6a09b995f3 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:19.661627 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7239be98-d49d-45e3-9533-31c762ad7c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:19.662773 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7239be98-d49d-45e3-9533-31c762ad7c7f admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:19.663319 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7239be98-d49d-45e3-9533-31c762ad7c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:19.664088 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7239be98-d49d-45e3-9533-31c762ad7c7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:19.666387 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7239be98-d49d-45e3-9533-31c762ad7c7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:19.668703 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7239be98-d49d-45e3-9533-31c762ad7c7f admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:19.695240 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8908f21-bc72-4dc7-8b17-baf89e9135ef admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:19.697909 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 868/1738] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:14:19 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 12:14:19.698806 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7239be98-d49d-45e3-9533-31c762ad7c7f admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:19.704382 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e16e80-3e9d-4b0c-aa92-dca92d9a20f1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7346e4a29dfb4a0a9768597704be286c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:19.705507 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e16e80-3e9d-4b0c-aa92-dca92d9a20f1 service placement] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:19.705798 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e16e80-3e9d-4b0c-aa92-dca92d9a20f1 service placement] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:19.706099 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e16e80-3e9d-4b0c-aa92-dca92d9a20f1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:19.708531 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-25e16e80-3e9d-4b0c-aa92-dca92d9a20f1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:19.714210 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7239be98-d49d-45e3-9533-31c762ad7c7f admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:19.720812 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f2098e-436c-42ce-9472-5113ee48cfb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:19.721430 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f2098e-436c-42ce-9472-5113ee48cfb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:19.721710 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f2098e-436c-42ce-9472-5113ee48cfb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:19.722028 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f2098e-436c-42ce-9472-5113ee48cfb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:19.722625 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f2098e-436c-42ce-9472-5113ee48cfb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:19.724258 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f2098e-436c-42ce-9472-5113ee48cfb2 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:19.725264 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e16e80-3e9d-4b0c-aa92-dca92d9a20f1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:19.728109 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e16e80-3e9d-4b0c-aa92-dca92d9a20f1 service placement] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:19.756812 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 869/1739] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:14:19 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:14:19.756812 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450b283-7d3f-4be2-8289-12c02efecb9e None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:19.756812 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450b283-7d3f-4be2-8289-12c02efecb9e None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:19.756812 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450b283-7d3f-4be2-8289-12c02efecb9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:19.779948 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 871/1740] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:14:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 12:14:19.781625 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e16e80-3e9d-4b0c-aa92-dca92d9a20f1 service placement] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:19.796444 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e16e80-3e9d-4b0c-aa92-dca92d9a20f1 service placement] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:19.801734 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696a349f-d1be-419b-89f4-d81c47e32509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:19.802465 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696a349f-d1be-419b-89f4-d81c47e32509 admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:19.802746 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696a349f-d1be-419b-89f4-d81c47e32509 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:19.803155 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696a349f-d1be-419b-89f4-d81c47e32509 admin admin] PATH_INFO: `/v3/projects/04fb0d30ff60483fa2c9be44753b59ca/users/534019974dea45ca8b6a35bff319c5d0/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:19.832365 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696a349f-d1be-419b-89f4-d81c47e32509 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04fb0d30ff60483fa2c9be44753b59ca, user_id=534019974dea45ca8b6a35bff319c5d0, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:19.839452 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696a349f-d1be-419b-89f4-d81c47e32509 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:19.852022 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-e450b283-7d3f-4be2-8289-12c02efecb9e None None] MFA Rules not processed for user `36a27ca98a2e4521b269be48c9ec4cb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:19.854409 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-e450b283-7d3f-4be2-8289-12c02efecb9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:19.890127 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:19.894711 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 872/1741] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:14:19 2022] PUT /identity/v3/projects/04fb0d30ff60483fa2c9be44753b59ca/users/534019974dea45ca8b6a35bff319c5d0/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:19.905971 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696a349f-d1be-419b-89f4-d81c47e32509 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:19.911326 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-696a349f-d1be-419b-89f4-d81c47e32509 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:20.050960 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:20.107669 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 870/1742] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:20.109104 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e008278-5f95-43e1-b23a-95247f87e46a None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:20.109399 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e008278-5f95-43e1-b23a-95247f87e46a None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:20.109674 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e008278-5f95-43e1-b23a-95247f87e46a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:20.128555 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696a349f-d1be-419b-89f4-d81c47e32509 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:20.144942 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab63370-21a4-49f5-a445-5b89767b0633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:20.145899 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab63370-21a4-49f5-a445-5b89767b0633 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:20.146289 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab63370-21a4-49f5-a445-5b89767b0633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:20.146671 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab63370-21a4-49f5-a445-5b89767b0633 admin admin] PATH_INFO: `/v3/projects/91f90a3555f04ec9b66a36f515926155` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:20.147855 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-5e008278-5f95-43e1-b23a-95247f87e46a None None] MFA Rules not processed for user `a02508c20bd547708638c1875101da6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:20.148644 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab63370-21a4-49f5-a445-5b89767b0633 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91f90a3555f04ec9b66a36f515926155)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:20.152505 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-5e008278-5f95-43e1-b23a-95247f87e46a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:20.161294 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab63370-21a4-49f5-a445-5b89767b0633 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:20.259038 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-fab63370-21a4-49f5-a445-5b89767b0633 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91f90a3555f04ec9b66a36f515926155', 'request_id': 'req-fab63370-21a4-49f5-a445-5b89767b0633'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:20.278980 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-fab63370-21a4-49f5-a445-5b89767b0633 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91f90a3555f04ec9b66a36f515926155', 'request_id': 'req-fab63370-21a4-49f5-a445-5b89767b0633'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:20.282231 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 871/1743] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:20 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:20.283395 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a04154-1f1f-4eb4-b9be-89bda70eb2d7 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:20.283684 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a04154-1f1f-4eb4-b9be-89bda70eb2d7 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:20.283977 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a04154-1f1f-4eb4-b9be-89bda70eb2d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:20.307644 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-fab63370-21a4-49f5-a445-5b89767b0633 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91f90a3555f04ec9b66a36f515926155', 'request_id': 'req-fab63370-21a4-49f5-a445-5b89767b0633'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:20.349632 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-fab63370-21a4-49f5-a445-5b89767b0633 admin admin] The token cache is being invalidate because project 91f90a3555f04ec9b66a36f515926155 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:14:20.350458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-f5a04154-1f1f-4eb4-b9be-89bda70eb2d7 None None] MFA Rules not processed for user `534019974dea45ca8b6a35bff319c5d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:20.350724 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-fab63370-21a4-49f5-a445-5b89767b0633 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:20.351909 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-f5a04154-1f1f-4eb4-b9be-89bda70eb2d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:20.352432 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-fab63370-21a4-49f5-a445-5b89767b0633 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:20.357599 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 873/1744] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:14:19 2022] DELETE /identity/v3/projects/91f90a3555f04ec9b66a36f515926155 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:20.366283 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab63370-21a4-49f5-a445-5b89767b0633 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:20.373262 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-fab63370-21a4-49f5-a445-5b89767b0633 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:20.510420 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 872/1745] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:20.511569 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a04154-1f1f-4eb4-b9be-89bda70eb2d7 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:20.513064 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-f5a04154-1f1f-4eb4-b9be-89bda70eb2d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:20.605927 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab63370-21a4-49f5-a445-5b89767b0633 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:20.620461 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:20.636857 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b240cf-0bc4-4946-8187-dda9b64c5e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:20.637526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b240cf-0bc4-4946-8187-dda9b64c5e60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:20.637807 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b240cf-0bc4-4946-8187-dda9b64c5e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:20.638082 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b240cf-0bc4-4946-8187-dda9b64c5e60 admin admin] PATH_INFO: `/v3/projects/d112018ddd254158ac5d36977c90374e` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:20.640419 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b240cf-0bc4-4946-8187-dda9b64c5e60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d112018ddd254158ac5d36977c90374e)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:20.644216 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b240cf-0bc4-4946-8187-dda9b64c5e60 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:20.753135 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a04154-1f1f-4eb4-b9be-89bda70eb2d7 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:20.781247 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-c4b240cf-0bc4-4946-8187-dda9b64c5e60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd112018ddd254158ac5d36977c90374e', 'request_id': 'req-c4b240cf-0bc4-4946-8187-dda9b64c5e60'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:20.800666 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-c4b240cf-0bc4-4946-8187-dda9b64c5e60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd112018ddd254158ac5d36977c90374e', 'request_id': 'req-c4b240cf-0bc4-4946-8187-dda9b64c5e60'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:20.822328 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-c4b240cf-0bc4-4946-8187-dda9b64c5e60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd112018ddd254158ac5d36977c90374e', 'request_id': 'req-c4b240cf-0bc4-4946-8187-dda9b64c5e60'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:20.829489 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:20.845660 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-c4b240cf-0bc4-4946-8187-dda9b64c5e60 admin admin] The token cache is being invalidate because project d112018ddd254158ac5d36977c90374e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:14:20.846114 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-c4b240cf-0bc4-4946-8187-dda9b64c5e60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:20.848926 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-c4b240cf-0bc4-4946-8187-dda9b64c5e60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:20.853729 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 874/1746] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:14:20 2022] DELETE /identity/v3/projects/d112018ddd254158ac5d36977c90374e => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:20.857663 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b240cf-0bc4-4946-8187-dda9b64c5e60 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:20.860713 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-c4b240cf-0bc4-4946-8187-dda9b64c5e60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:20.937074 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cd95a2-4331-4730-bba2-035b5ff48968 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aacd7d6d17e4f10ba5c9107a12beafa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:20.938059 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd95a2-4331-4730-bba2-035b5ff48968 service nova] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:20.938059 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd95a2-4331-4730-bba2-035b5ff48968 service nova] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:20.938236 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd95a2-4331-4730-bba2-035b5ff48968 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:20.951080 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-42cd95a2-4331-4730-bba2-035b5ff48968 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:21.112477 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b240cf-0bc4-4946-8187-dda9b64c5e60 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:21.125959 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e34efc-4c70-4c9b-bbb1-25bfdde5093f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:21.127025 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e34efc-4c70-4c9b-bbb1-25bfdde5093f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:21.127828 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e34efc-4c70-4c9b-bbb1-25bfdde5093f admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:21.128530 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e34efc-4c70-4c9b-bbb1-25bfdde5093f admin admin] PATH_INFO: `/v3/users/64ea2f60612b44e1b2c0c7b48ff61517` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:21.130792 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e34efc-4c70-4c9b-bbb1-25bfdde5093f admin admin] RBAC: Authorizing `identity:delete_user(user_id=64ea2f60612b44e1b2c0c7b48ff61517)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:21.146546 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e34efc-4c70-4c9b-bbb1-25bfdde5093f admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:21.159454 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cd95a2-4331-4730-bba2-035b5ff48968 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:21.161072 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cd95a2-4331-4730-bba2-035b5ff48968 service nova] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:21.186664 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 873/1747] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:14:20 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 676 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:14:21.188125 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e77967-2172-4993-a75b-a5180ae7e9da None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:21.188419 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e77967-2172-4993-a75b-a5180ae7e9da None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:21.188702 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e77967-2172-4993-a75b-a5180ae7e9da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:21.265709 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-f1e34efc-4c70-4c9b-bbb1-25bfdde5093f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64ea2f60612b44e1b2c0c7b48ff61517'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:21.276573 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-f1e34efc-4c70-4c9b-bbb1-25bfdde5093f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64ea2f60612b44e1b2c0c7b48ff61517'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:21.289642 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-47e77967-2172-4993-a75b-a5180ae7e9da None None] MFA Rules not processed for user `ff96b716abcd4565866fc3c24cbe91e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:21.292804 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-47e77967-2172-4993-a75b-a5180ae7e9da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:21.293863 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-f1e34efc-4c70-4c9b-bbb1-25bfdde5093f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64ea2f60612b44e1b2c0c7b48ff61517'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:21.295061 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-f1e34efc-4c70-4c9b-bbb1-25bfdde5093f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64ea2f60612b44e1b2c0c7b48ff61517'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:21.324296 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-f1e34efc-4c70-4c9b-bbb1-25bfdde5093f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64ea2f60612b44e1b2c0c7b48ff61517'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:21.346123 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 875/1748] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:14:20 2022] DELETE /identity/v3/users/64ea2f60612b44e1b2c0c7b48ff61517 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:21.346123 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e34efc-4c70-4c9b-bbb1-25bfdde5093f admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:21.348340 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-f1e34efc-4c70-4c9b-bbb1-25bfdde5093f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:21.448745 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e34efc-4c70-4c9b-bbb1-25bfdde5093f admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:21.454381 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa18f7ee-f227-4569-8a66-2f9a9c27d5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:21.455060 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa18f7ee-f227-4569-8a66-2f9a9c27d5a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:21.455377 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa18f7ee-f227-4569-8a66-2f9a9c27d5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:21.455657 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa18f7ee-f227-4569-8a66-2f9a9c27d5a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:21.456662 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa18f7ee-f227-4569-8a66-2f9a9c27d5a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:21.463296 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa18f7ee-f227-4569-8a66-2f9a9c27d5a2 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:21.483296 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 874/1749] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:21.495897 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb6fffa-6f75-444e-9aee-8c6b1ed2e135 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:21.496257 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb6fffa-6f75-444e-9aee-8c6b1ed2e135 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:21.496613 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb6fffa-6f75-444e-9aee-8c6b1ed2e135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:21.513915 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:21.517091 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:21.531133 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-abb6fffa-6f75-444e-9aee-8c6b1ed2e135 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:21.531474 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 876/1750] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:14:21 2022] POST /identity/v3/projects => generated 367 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:14:21.534430 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-abb6fffa-6f75-444e-9aee-8c6b1ed2e135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:21.534579 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99369276-650f-43dd-b1ae-04816ff2c6be None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:21.537778 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99369276-650f-43dd-b1ae-04816ff2c6be None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:21.539318 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99369276-650f-43dd-b1ae-04816ff2c6be None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:21.548093 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 877/1751] 10.222.0.61 () {54 vars in 912 bytes} [Fri Aug 26 12:14:21 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:14:21.550178 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7189a6-2803-4049-958f-f9cad8f6f666 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:21.550470 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7189a6-2803-4049-958f-f9cad8f6f666 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:21.552793 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7189a6-2803-4049-958f-f9cad8f6f666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:21.606941 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-eb7189a6-2803-4049-958f-f9cad8f6f666 None None] MFA Rules not processed for user `ff96b716abcd4565866fc3c24cbe91e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:21.610636 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-eb7189a6-2803-4049-958f-f9cad8f6f666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:21.692146 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 875/1752] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:21.693082 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb6fffa-6f75-444e-9aee-8c6b1ed2e135 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:21.721675 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb6fffa-6f75-444e-9aee-8c6b1ed2e135 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:21.763675 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34669e73-ea5c-4629-b290-f7e307b83e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:21.765025 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34669e73-ea5c-4629-b290-f7e307b83e76 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:21.765313 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34669e73-ea5c-4629-b290-f7e307b83e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:21.765599 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34669e73-ea5c-4629-b290-f7e307b83e76 admin admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:21.767663 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34669e73-ea5c-4629-b290-f7e307b83e76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:21.771185 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 878/1753] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:21.774776 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25724324-e1ee-48b9-80fd-5849a97ab0ba None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:21.776286 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25724324-e1ee-48b9-80fd-5849a97ab0ba None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:21.776709 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25724324-e1ee-48b9-80fd-5849a97ab0ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:21.795554 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34669e73-ea5c-4629-b290-f7e307b83e76 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:21.820645 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-25724324-e1ee-48b9-80fd-5849a97ab0ba None None] MFA Rules not processed for user `b0972dbd5cff4610b239326425e6351c`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:21.822330 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-25724324-e1ee-48b9-80fd-5849a97ab0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:21.843513 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 876/1754] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:14:21 2022] POST /identity/v3/users => generated 290 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:14:21.846151 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4632b655-88d4-4e4a-a74e-c7415d1a61da None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:21.846703 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4632b655-88d4-4e4a-a74e-c7415d1a61da None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:21.847117 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4632b655-88d4-4e4a-a74e-c7415d1a61da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:21.963253 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-4632b655-88d4-4e4a-a74e-c7415d1a61da None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:21.966124 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-4632b655-88d4-4e4a-a74e-c7415d1a61da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:21.980997 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 879/1755] 10.222.0.61 () {58 vars in 1009 bytes} [Fri Aug 26 12:14:21 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:21.982973 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25724324-e1ee-48b9-80fd-5849a97ab0ba None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:21.994938 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-25724324-e1ee-48b9-80fd-5849a97ab0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:22.004768 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 877/1756] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:22.006004 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4632b655-88d4-4e4a-a74e-c7415d1a61da None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:22.028995 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4632b655-88d4-4e4a-a74e-c7415d1a61da None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:22.040386 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa8f7eb-01b7-4aca-a451-f5d537c31d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:22.041298 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8f7eb-01b7-4aca-a451-f5d537c31d86 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:22.042288 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8f7eb-01b7-4aca-a451-f5d537c31d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:22.043537 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8f7eb-01b7-4aca-a451-f5d537c31d86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:22.044484 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa8f7eb-01b7-4aca-a451-f5d537c31d86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:22.047873 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa8f7eb-01b7-4aca-a451-f5d537c31d86 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:22.064708 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 878/1757] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:14:22 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:14:22.067914 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa8f7eb-01b7-4aca-a451-f5d537c31d86 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:22.087714 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa8f7eb-01b7-4aca-a451-f5d537c31d86 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:22.114750 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10bd893-acfd-49da-a383-7fdae90eff1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:22.115746 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10bd893-acfd-49da-a383-7fdae90eff1a admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:22.115977 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10bd893-acfd-49da-a383-7fdae90eff1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:22.116245 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10bd893-acfd-49da-a383-7fdae90eff1a admin admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:22.116900 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10bd893-acfd-49da-a383-7fdae90eff1a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:14:22.117201 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10bd893-acfd-49da-a383-7fdae90eff1a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:22.119123 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10bd893-acfd-49da-a383-7fdae90eff1a admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:22.155898 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 879/1758] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:14:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:14:22.158791 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10bd893-acfd-49da-a383-7fdae90eff1a admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:22.189938 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10bd893-acfd-49da-a383-7fdae90eff1a admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:22.204402 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be303ce8-dc50-45fb-90da-1ab866e21101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:22.205243 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be303ce8-dc50-45fb-90da-1ab866e21101 admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:22.205528 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be303ce8-dc50-45fb-90da-1ab866e21101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:22.207760 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be303ce8-dc50-45fb-90da-1ab866e21101 admin admin] PATH_INFO: `/v3/projects/bd793d6416e246cc90bd4ebc35a235e1/users/c02fb133772c407593d649e82b4252a3/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:22.220837 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25724324-e1ee-48b9-80fd-5849a97ab0ba None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:22.238748 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850dd550-29f9-460b-88f8-c3ca93a77438 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aacd7d6d17e4f10ba5c9107a12beafa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:22.240947 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850dd550-29f9-460b-88f8-c3ca93a77438 service nova] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:22.241232 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850dd550-29f9-460b-88f8-c3ca93a77438 service nova] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:22.241507 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850dd550-29f9-460b-88f8-c3ca93a77438 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:22.247788 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be303ce8-dc50-45fb-90da-1ab866e21101 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd793d6416e246cc90bd4ebc35a235e1, user_id=c02fb133772c407593d649e82b4252a3, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:22.254002 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be303ce8-dc50-45fb-90da-1ab866e21101 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:22.265436 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850dd550-29f9-460b-88f8-c3ca93a77438 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:22.268606 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850dd550-29f9-460b-88f8-c3ca93a77438 service nova] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:22.289364 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 880/1759] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:14:22 2022] PUT /identity/v3/projects/bd793d6416e246cc90bd4ebc35a235e1/users/c02fb133772c407593d649e82b4252a3/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:22.290907 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be303ce8-dc50-45fb-90da-1ab866e21101 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:22.307576 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-be303ce8-dc50-45fb-90da-1ab866e21101 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:22.311743 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 880/1760] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:14:21 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:14:22.312854 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850dd550-29f9-460b-88f8-c3ca93a77438 service nova] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:22.356548 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850dd550-29f9-460b-88f8-c3ca93a77438 service nova] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:22.523652 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:22.529037 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be303ce8-dc50-45fb-90da-1ab866e21101 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:22.541202 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8443002-e7c4-410f-9e50-7d30d1f5cee9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:22.542878 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8443002-e7c4-410f-9e50-7d30d1f5cee9 service neutron] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:22.543622 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8443002-e7c4-410f-9e50-7d30d1f5cee9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:22.545588 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8443002-e7c4-410f-9e50-7d30d1f5cee9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:22.580735 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8443002-e7c4-410f-9e50-7d30d1f5cee9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:22.582316 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8443002-e7c4-410f-9e50-7d30d1f5cee9 service neutron] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:22.601088 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179bd54f-315a-4d4e-96a3-679458c91e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:22.601722 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179bd54f-315a-4d4e-96a3-679458c91e61 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:22.602038 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179bd54f-315a-4d4e-96a3-679458c91e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:22.602313 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179bd54f-315a-4d4e-96a3-679458c91e61 admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:22.603043 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179bd54f-315a-4d4e-96a3-679458c91e61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:22.605120 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179bd54f-315a-4d4e-96a3-679458c91e61 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:22.625951 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 881/1761] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:14:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:14:22.625951 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8443002-e7c4-410f-9e50-7d30d1f5cee9 service neutron] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:22.675298 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8443002-e7c4-410f-9e50-7d30d1f5cee9 service neutron] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:22.691680 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34890ec9-d7b3-4529-83a0-1f0c6d7001f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:22.694126 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34890ec9-d7b3-4529-83a0-1f0c6d7001f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:22.694420 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34890ec9-d7b3-4529-83a0-1f0c6d7001f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:22.704235 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34890ec9-d7b3-4529-83a0-1f0c6d7001f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:22.705030 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34890ec9-d7b3-4529-83a0-1f0c6d7001f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:22.706804 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34890ec9-d7b3-4529-83a0-1f0c6d7001f6 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:22.719680 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 881/1762] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:14:22 2022] POST /identity/v3/projects => generated 377 bytes in 407 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:14:22.720386 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179bd54f-315a-4d4e-96a3-679458c91e61 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:22.743727 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-179bd54f-315a-4d4e-96a3-679458c91e61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:22.765004 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 882/1763] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:14:22 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:14:22.768804 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34890ec9-d7b3-4529-83a0-1f0c6d7001f6 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:22.809781 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34890ec9-d7b3-4529-83a0-1f0c6d7001f6 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:22.916611 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:22.978115 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179bd54f-315a-4d4e-96a3-679458c91e61 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:22.991048 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b174c3eb-a9f3-4b9a-a5c2-ad2032df2931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:22.992512 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b174c3eb-a9f3-4b9a-a5c2-ad2032df2931 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:22.993785 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b174c3eb-a9f3-4b9a-a5c2-ad2032df2931 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:22.994965 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b174c3eb-a9f3-4b9a-a5c2-ad2032df2931 admin admin] PATH_INFO: `/v3/projects/dbd0751749af4d16ad697394257ba9c5` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:22.995995 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c81fd9-efab-44ba-a734-ad88ab3583b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:22.997494 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b174c3eb-a9f3-4b9a-a5c2-ad2032df2931 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbd0751749af4d16ad697394257ba9c5)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:22.997700 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c81fd9-efab-44ba-a734-ad88ab3583b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:22.998651 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c81fd9-efab-44ba-a734-ad88ab3583b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:22.999075 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c81fd9-efab-44ba-a734-ad88ab3583b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:23.000421 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c81fd9-efab-44ba-a734-ad88ab3583b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:23.003381 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c81fd9-efab-44ba-a734-ad88ab3583b4 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:23.019293 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b174c3eb-a9f3-4b9a-a5c2-ad2032df2931 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:23.054757 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 883/1764] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:14:22 2022] POST /identity/v3/users => generated 289 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:14:23.059607 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c81fd9-efab-44ba-a734-ad88ab3583b4 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:23.089768 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c81fd9-efab-44ba-a734-ad88ab3583b4 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:23.098841 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90eb774-f9da-4777-b39b-8e76fd20f681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:23.099725 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90eb774-f9da-4777-b39b-8e76fd20f681 admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:23.100114 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90eb774-f9da-4777-b39b-8e76fd20f681 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:23.100496 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90eb774-f9da-4777-b39b-8e76fd20f681 admin admin] PATH_INFO: `/v3/projects/bd793d6416e246cc90bd4ebc35a235e1/users/c02fb133772c407593d649e82b4252a3/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:23.107940 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90eb774-f9da-4777-b39b-8e76fd20f681 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd793d6416e246cc90bd4ebc35a235e1, user_id=c02fb133772c407593d649e82b4252a3, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:23.110903 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90eb774-f9da-4777-b39b-8e76fd20f681 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:23.123735 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-b174c3eb-a9f3-4b9a-a5c2-ad2032df2931 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbd0751749af4d16ad697394257ba9c5', 'request_id': 'req-b174c3eb-a9f3-4b9a-a5c2-ad2032df2931'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:23.133681 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-b174c3eb-a9f3-4b9a-a5c2-ad2032df2931 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbd0751749af4d16ad697394257ba9c5', 'request_id': 'req-b174c3eb-a9f3-4b9a-a5c2-ad2032df2931'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:23.144569 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 884/1765] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:14:23 2022] PUT /identity/v3/projects/bd793d6416e246cc90bd4ebc35a235e1/users/c02fb133772c407593d649e82b4252a3/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:23.144569 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61778fb9-3e10-40ad-844e-75c18c264481 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:23.144569 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61778fb9-3e10-40ad-844e-75c18c264481 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:23.144569 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61778fb9-3e10-40ad-844e-75c18c264481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:23.157193 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-b174c3eb-a9f3-4b9a-a5c2-ad2032df2931 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbd0751749af4d16ad697394257ba9c5', 'request_id': 'req-b174c3eb-a9f3-4b9a-a5c2-ad2032df2931'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:23.176620 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-b174c3eb-a9f3-4b9a-a5c2-ad2032df2931 admin admin] The token cache is being invalidate because project dbd0751749af4d16ad697394257ba9c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:14:23.176910 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-b174c3eb-a9f3-4b9a-a5c2-ad2032df2931 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:23.177931 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-b174c3eb-a9f3-4b9a-a5c2-ad2032df2931 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:23.179380 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 882/1766] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:14:22 2022] DELETE /identity/v3/projects/dbd0751749af4d16ad697394257ba9c5 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:23.180820 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-61778fb9-3e10-40ad-844e-75c18c264481 None None] MFA Rules not processed for user `2d9f1a5e4e77467f8a691b634a841de2`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:23.183721 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-61778fb9-3e10-40ad-844e-75c18c264481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:23.184057 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b174c3eb-a9f3-4b9a-a5c2-ad2032df2931 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:23.195080 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-b174c3eb-a9f3-4b9a-a5c2-ad2032df2931 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:23.334638 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:23.344926 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 885/1767] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:23.349089 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbb4174-d0bf-4d63-8392-a82c8a33e98d None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:23.349687 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbb4174-d0bf-4d63-8392-a82c8a33e98d None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:23.351036 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbb4174-d0bf-4d63-8392-a82c8a33e98d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:23.354159 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b174c3eb-a9f3-4b9a-a5c2-ad2032df2931 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:23.426264 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-7fbb4174-d0bf-4d63-8392-a82c8a33e98d None None] MFA Rules not processed for user `c02fb133772c407593d649e82b4252a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:23.428039 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-7fbb4174-d0bf-4d63-8392-a82c8a33e98d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:23.515437 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c19afb3-f21f-4f67-a875-c8cfc69b71e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:23.516108 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c19afb3-f21f-4f67-a875-c8cfc69b71e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:23.516389 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c19afb3-f21f-4f67-a875-c8cfc69b71e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:23.516675 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c19afb3-f21f-4f67-a875-c8cfc69b71e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:23.517306 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c19afb3-f21f-4f67-a875-c8cfc69b71e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:23.518918 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c19afb3-f21f-4f67-a875-c8cfc69b71e0 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:23.547737 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 883/1768] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:14:23 2022] GET /identity/v3/roles => generated 1420 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:14:23.549288 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcbacd3-2014-440a-927c-37e01e9f31be None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:23.549568 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcbacd3-2014-440a-927c-37e01e9f31be None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:23.549859 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcbacd3-2014-440a-927c-37e01e9f31be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:23.594528 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 886/1769] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:23 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:23.596213 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbb4174-d0bf-4d63-8392-a82c8a33e98d None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:23.605485 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-8fcbacd3-2014-440a-927c-37e01e9f31be None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:23.607748 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-8fcbacd3-2014-440a-927c-37e01e9f31be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:23.615160 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbb4174-d0bf-4d63-8392-a82c8a33e98d None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:23.630240 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84789ee6-199e-4ade-a581-44e1e5040973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:23.631582 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 884/1770] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:23.633153 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84789ee6-199e-4ade-a581-44e1e5040973 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:23.634219 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcbacd3-2014-440a-927c-37e01e9f31be None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:23.636031 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84789ee6-199e-4ade-a581-44e1e5040973 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:23.636327 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84789ee6-199e-4ade-a581-44e1e5040973 admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:23.637208 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84789ee6-199e-4ade-a581-44e1e5040973 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:23.639219 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84789ee6-199e-4ade-a581-44e1e5040973 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:23.644228 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-8fcbacd3-2014-440a-927c-37e01e9f31be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:23.656107 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 887/1771] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:14:23 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:14:23.663108 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1595f8-df0a-4f7f-a169-1e5b0967023c None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:23.663728 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1595f8-df0a-4f7f-a169-1e5b0967023c None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:23.664373 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1595f8-df0a-4f7f-a169-1e5b0967023c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:23.694815 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-9c1595f8-df0a-4f7f-a169-1e5b0967023c None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:23.697608 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-9c1595f8-df0a-4f7f-a169-1e5b0967023c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:23.711473 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 888/1772] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:23.711473 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1595f8-df0a-4f7f-a169-1e5b0967023c None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:23.724596 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1595f8-df0a-4f7f-a169-1e5b0967023c None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:23.732703 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f381e30a-078d-46ff-a83a-fd57063b03c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:23.733650 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f381e30a-078d-46ff-a83a-fd57063b03c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:23.734043 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f381e30a-078d-46ff-a83a-fd57063b03c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:23.734434 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f381e30a-078d-46ff-a83a-fd57063b03c3 admin admin] PATH_INFO: `/v3/projects/064ad4d5e69d4ac4aaa35ddbeead2cb7/users/e918f83f201c47faaf8355111ce74914/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:23.752230 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:23.757318 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f381e30a-078d-46ff-a83a-fd57063b03c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=064ad4d5e69d4ac4aaa35ddbeead2cb7, user_id=e918f83f201c47faaf8355111ce74914, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:23.763640 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f381e30a-078d-46ff-a83a-fd57063b03c3 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:23.787523 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 889/1773] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:14:23 2022] PUT /identity/v3/projects/064ad4d5e69d4ac4aaa35ddbeead2cb7/users/e918f83f201c47faaf8355111ce74914/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:23.788225 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcbacd3-2014-440a-927c-37e01e9f31be None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:23.816646 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f381e30a-078d-46ff-a83a-fd57063b03c3 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:23.941078 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64917245-568d-4e32-84f1-99f0c754bcf2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:23.941785 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64917245-568d-4e32-84f1-99f0c754bcf2 service cinder] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:23.942069 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64917245-568d-4e32-84f1-99f0c754bcf2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:23.942349 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64917245-568d-4e32-84f1-99f0c754bcf2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:24.097169 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f381e30a-078d-46ff-a83a-fd57063b03c3 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:24.119462 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e31db35-f40e-4d97-a89b-35e5ce6ddd89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:24.120295 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31db35-f40e-4d97-a89b-35e5ce6ddd89 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:24.120910 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31db35-f40e-4d97-a89b-35e5ce6ddd89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:24.121025 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31db35-f40e-4d97-a89b-35e5ce6ddd89 admin admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:24.121770 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e31db35-f40e-4d97-a89b-35e5ce6ddd89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:14:24.122178 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e31db35-f40e-4d97-a89b-35e5ce6ddd89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:24.124068 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e31db35-f40e-4d97-a89b-35e5ce6ddd89 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:24.190441 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64917245-568d-4e32-84f1-99f0c754bcf2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:24.192265 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64917245-568d-4e32-84f1-99f0c754bcf2 service cinder] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:24.200421 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 890/1774] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:14:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:14:24.210746 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ce645-926b-4a03-a2d1-3e6017328eaa None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:24.211039 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ce645-926b-4a03-a2d1-3e6017328eaa None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:24.222898 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ce645-926b-4a03-a2d1-3e6017328eaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:24.225413 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:24.245427 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 885/1775] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:14:23 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 612 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:14:24.246563 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64917245-568d-4e32-84f1-99f0c754bcf2 service cinder] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:24.297513 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64917245-568d-4e32-84f1-99f0c754bcf2 service cinder] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:24.359021 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a888274-9b48-4207-97b8-703f9c119af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:24.359021 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a888274-9b48-4207-97b8-703f9c119af4 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:24.359021 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a888274-9b48-4207-97b8-703f9c119af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:24.359433 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a888274-9b48-4207-97b8-703f9c119af4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:24.361690 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-d97ce645-926b-4a03-a2d1-3e6017328eaa None None] MFA Rules not processed for user `e918f83f201c47faaf8355111ce74914`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:24.366167 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-d97ce645-926b-4a03-a2d1-3e6017328eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:24.378035 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a888274-9b48-4207-97b8-703f9c119af4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:24.383903 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a888274-9b48-4207-97b8-703f9c119af4 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:24.638841 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 886/1776] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:14:24 2022] POST /identity/v3/projects => generated 387 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:14:24.642845 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a888274-9b48-4207-97b8-703f9c119af4 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:24.671800 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 891/1777] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 471 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:24.672688 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97ce645-926b-4a03-a2d1-3e6017328eaa None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:24.726847 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97ce645-926b-4a03-a2d1-3e6017328eaa None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:24.878317 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a888274-9b48-4207-97b8-703f9c119af4 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:24.909918 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab172265-3dc8-4dc9-a9df-65ad54c41bf9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:24.911677 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab172265-3dc8-4dc9-a9df-65ad54c41bf9 service cinder] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:24.912716 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab172265-3dc8-4dc9-a9df-65ad54c41bf9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:24.916558 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab172265-3dc8-4dc9-a9df-65ad54c41bf9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:24.935646 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-ab172265-3dc8-4dc9-a9df-65ad54c41bf9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:25.055350 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f5c108-ed01-45fa-add8-0dcf4862e3dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:25.057811 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f5c108-ed01-45fa-add8-0dcf4862e3dc admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:25.058394 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f5c108-ed01-45fa-add8-0dcf4862e3dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:25.059591 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f5c108-ed01-45fa-add8-0dcf4862e3dc admin admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:25.067284 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f5c108-ed01-45fa-add8-0dcf4862e3dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:25.077749 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f5c108-ed01-45fa-add8-0dcf4862e3dc admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:25.171334 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 892/1778] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:14:24 2022] POST /identity/v3/users => generated 294 bytes in 497 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:14:25.172376 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f5c108-ed01-45fa-add8-0dcf4862e3dc admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:25.195238 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:25.204509 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f5c108-ed01-45fa-add8-0dcf4862e3dc admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:25.227784 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9ebdf9-6d84-4ec6-845a-b033fc81285d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:25.230101 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab172265-3dc8-4dc9-a9df-65ad54c41bf9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:25.234948 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9ebdf9-6d84-4ec6-845a-b033fc81285d admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:25.236087 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab172265-3dc8-4dc9-a9df-65ad54c41bf9 service cinder] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:25.236469 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9ebdf9-6d84-4ec6-845a-b033fc81285d admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:25.236845 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9ebdf9-6d84-4ec6-845a-b033fc81285d admin admin] PATH_INFO: `/v3/projects` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:25.239196 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9ebdf9-6d84-4ec6-845a-b033fc81285d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:25.244655 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9ebdf9-6d84-4ec6-845a-b033fc81285d admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:25.270015 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 887/1779] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:14:24 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 630 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:14:25.288722 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5435f3-a550-4feb-b771-9b088076faf7 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:25.289318 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5435f3-a550-4feb-b771-9b088076faf7 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:25.292842 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5435f3-a550-4feb-b771-9b088076faf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:25.369908 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 893/1780] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:14:25 2022] POST /identity/v3/projects => generated 381 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:14:25.369908 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9ebdf9-6d84-4ec6-845a-b033fc81285d admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:25.509285 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-2c5435f3-a550-4feb-b771-9b088076faf7 None None] MFA Rules not processed for user `755b04cae45746569bfbf92f2f213a58`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:25.515754 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-2c5435f3-a550-4feb-b771-9b088076faf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:25.646385 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9ebdf9-6d84-4ec6-845a-b033fc81285d admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:25.668123 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64af1b8d-6428-440b-945d-e07f7cec36fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:25.670519 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af1b8d-6428-440b-945d-e07f7cec36fe admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:25.670519 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af1b8d-6428-440b-945d-e07f7cec36fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:25.676484 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af1b8d-6428-440b-945d-e07f7cec36fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:25.677866 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64af1b8d-6428-440b-945d-e07f7cec36fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:25.682635 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64af1b8d-6428-440b-945d-e07f7cec36fe admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:25.712143 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 894/1781] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:14:25 2022] GET /identity/v3/roles => generated 1420 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:14:25.721068 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64af1b8d-6428-440b-945d-e07f7cec36fe admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:25.756807 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64af1b8d-6428-440b-945d-e07f7cec36fe admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:25.783943 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d7d4d-8c3a-465d-80a0-872e7ff7f2f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:25.786228 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d7d4d-8c3a-465d-80a0-872e7ff7f2f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:25.788734 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d7d4d-8c3a-465d-80a0-872e7ff7f2f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:25.789125 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d7d4d-8c3a-465d-80a0-872e7ff7f2f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:25.790293 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 888/1782] 10.222.0.61 () {58 vars in 1009 bytes} [Fri Aug 26 12:14:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 508 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:25.793128 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26d7d4d-8c3a-465d-80a0-872e7ff7f2f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:25.795149 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5435f3-a550-4feb-b771-9b088076faf7 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:25.800652 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26d7d4d-8c3a-465d-80a0-872e7ff7f2f3 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:25.820462 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:25.870723 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5435f3-a550-4feb-b771-9b088076faf7 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:25.890908 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07152e06-3aef-4479-8eaf-9a589f097594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:25.891961 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07152e06-3aef-4479-8eaf-9a589f097594 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:25.892649 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07152e06-3aef-4479-8eaf-9a589f097594 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:25.893124 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07152e06-3aef-4479-8eaf-9a589f097594 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:25.894263 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07152e06-3aef-4479-8eaf-9a589f097594 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:25.896479 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07152e06-3aef-4479-8eaf-9a589f097594 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:25.913734 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 895/1783] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:14:25 2022] POST /identity/v3/users => generated 297 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:14:25.918615 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d7d4d-8c3a-465d-80a0-872e7ff7f2f3 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:25.950552 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 889/1784] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:14:25 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:14:25.955028 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07152e06-3aef-4479-8eaf-9a589f097594 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:25.956847 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d7d4d-8c3a-465d-80a0-872e7ff7f2f3 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:25.968053 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4476e3-d1c7-4be8-9a64-fee08852abdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:25.969039 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4476e3-d1c7-4be8-9a64-fee08852abdf admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:25.969663 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4476e3-d1c7-4be8-9a64-fee08852abdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:25.970164 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4476e3-d1c7-4be8-9a64-fee08852abdf admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:25.971116 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4476e3-d1c7-4be8-9a64-fee08852abdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:25.973225 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4476e3-d1c7-4be8-9a64-fee08852abdf admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:25.989511 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07152e06-3aef-4479-8eaf-9a589f097594 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:26.014693 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 896/1785] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:14:25 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:14:26.015994 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4476e3-d1c7-4be8-9a64-fee08852abdf admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:26.020518 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed218f2b-f1ed-47f7-9f6d-7aa2dbaa1fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:26.021464 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed218f2b-f1ed-47f7-9f6d-7aa2dbaa1fcf admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:26.021544 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed218f2b-f1ed-47f7-9f6d-7aa2dbaa1fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:26.021770 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed218f2b-f1ed-47f7-9f6d-7aa2dbaa1fcf admin admin] PATH_INFO: `/v3/projects/be9334cf7b3a45939604edc1c9f2eeef/users/97b4d37ce3354f1d907607fb0a1de35e/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:26.040361 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4476e3-d1c7-4be8-9a64-fee08852abdf admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:26.046213 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abd94ef-45a7-460a-8d62-647b6e4c0ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:26.061156 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abd94ef-45a7-460a-8d62-647b6e4c0ae4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:26.061618 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abd94ef-45a7-460a-8d62-647b6e4c0ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:26.062679 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abd94ef-45a7-460a-8d62-647b6e4c0ae4 admin admin] PATH_INFO: `/v3/projects/0ecedd70b8d043668a6b7b01cbd6ee85/users/27c5678aa26644629197b10a46518d07/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:26.074473 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed218f2b-f1ed-47f7-9f6d-7aa2dbaa1fcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=be9334cf7b3a45939604edc1c9f2eeef, user_id=97b4d37ce3354f1d907607fb0a1de35e, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:26.087337 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed218f2b-f1ed-47f7-9f6d-7aa2dbaa1fcf admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:26.160854 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abd94ef-45a7-460a-8d62-647b6e4c0ae4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ecedd70b8d043668a6b7b01cbd6ee85, user_id=27c5678aa26644629197b10a46518d07, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:26.170150 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 890/1786] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:14:25 2022] PUT /identity/v3/projects/be9334cf7b3a45939604edc1c9f2eeef/users/97b4d37ce3354f1d907607fb0a1de35e/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:26.171264 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed218f2b-f1ed-47f7-9f6d-7aa2dbaa1fcf admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:26.186443 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-ed218f2b-f1ed-47f7-9f6d-7aa2dbaa1fcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:26.189383 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abd94ef-45a7-460a-8d62-647b6e4c0ae4 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:26.239461 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 897/1787] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:14:26 2022] PUT /identity/v3/projects/0ecedd70b8d043668a6b7b01cbd6ee85/users/27c5678aa26644629197b10a46518d07/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:26.240889 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f1a97-1714-4cd8-8de6-5687ca383079 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:26.241273 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f1a97-1714-4cd8-8de6-5687ca383079 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:26.241651 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f1a97-1714-4cd8-8de6-5687ca383079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:26.331360 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-1a5f1a97-1714-4cd8-8de6-5687ca383079 None None] MFA Rules not processed for user `97b4d37ce3354f1d907607fb0a1de35e`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:26.334008 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-1a5f1a97-1714-4cd8-8de6-5687ca383079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:26.504093 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed218f2b-f1ed-47f7-9f6d-7aa2dbaa1fcf admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:26.506795 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 898/1788] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:26 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:26.506795 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5f1a97-1714-4cd8-8de6-5687ca383079 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:26.582547 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:26.663540 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea38c18-9e23-4242-af0c-1cb4dd0f8e98 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead95a244d11432d9fe9f718e1a75f38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:26.665575 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea38c18-9e23-4242-af0c-1cb4dd0f8e98 service swift] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:26.666070 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea38c18-9e23-4242-af0c-1cb4dd0f8e98 service swift] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:26.667162 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea38c18-9e23-4242-af0c-1cb4dd0f8e98 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:26.726182 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5f1a97-1714-4cd8-8de6-5687ca383079 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:26.758435 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c166ece4-9acb-4e89-9d49-eb9dbdfebc5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:26.759319 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c166ece4-9acb-4e89-9d49-eb9dbdfebc5f admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:26.759592 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c166ece4-9acb-4e89-9d49-eb9dbdfebc5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:26.759859 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c166ece4-9acb-4e89-9d49-eb9dbdfebc5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:26.760782 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c166ece4-9acb-4e89-9d49-eb9dbdfebc5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:26.762417 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c166ece4-9acb-4e89-9d49-eb9dbdfebc5f admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:26.798228 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 899/1789] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:14:26.798228 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c166ece4-9acb-4e89-9d49-eb9dbdfebc5f admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:26.814763 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c166ece4-9acb-4e89-9d49-eb9dbdfebc5f admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:26.841198 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95193c0-7ae9-47b7-819a-9c47bf8be9d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:26.841198 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95193c0-7ae9-47b7-819a-9c47bf8be9d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:26.841198 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95193c0-7ae9-47b7-819a-9c47bf8be9d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:26.841198 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95193c0-7ae9-47b7-819a-9c47bf8be9d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:26.841736 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95193c0-7ae9-47b7-819a-9c47bf8be9d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:26.841736 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95193c0-7ae9-47b7-819a-9c47bf8be9d1 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:26.947027 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:26.951196 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 900/1790] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:14:26 2022] POST /identity/v3/projects => generated 387 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:14:26.951196 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95193c0-7ae9-47b7-819a-9c47bf8be9d1 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:27.001839 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea38c18-9e23-4242-af0c-1cb4dd0f8e98 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:27.014151 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea38c18-9e23-4242-af0c-1cb4dd0f8e98 service swift] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:27.054292 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 891/1791] 10.222.0.61 () {60 vars in 1384 bytes} [Fri Aug 26 12:14:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 883 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 12:14:27.058723 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea38c18-9e23-4242-af0c-1cb4dd0f8e98 service swift] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:27.112979 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea38c18-9e23-4242-af0c-1cb4dd0f8e98 service swift] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:27.126144 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cdaf32-4658-429b-aa67-8664a5a3a1a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:27.127325 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cdaf32-4658-429b-aa67-8664a5a3a1a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:27.127864 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cdaf32-4658-429b-aa67-8664a5a3a1a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:27.128378 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cdaf32-4658-429b-aa67-8664a5a3a1a0 admin admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:27.129141 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cdaf32-4658-429b-aa67-8664a5a3a1a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:27.131248 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cdaf32-4658-429b-aa67-8664a5a3a1a0 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:27.148001 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95193c0-7ae9-47b7-819a-9c47bf8be9d1 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:27.170610 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef69b7d3-374b-450a-8fd0-50f41a99447a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:27.171375 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef69b7d3-374b-450a-8fd0-50f41a99447a admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:27.171660 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef69b7d3-374b-450a-8fd0-50f41a99447a admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:27.171941 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef69b7d3-374b-450a-8fd0-50f41a99447a admin admin] PATH_INFO: `/v3/projects/0ecedd70b8d043668a6b7b01cbd6ee85/users/27c5678aa26644629197b10a46518d07/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:27.183187 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef69b7d3-374b-450a-8fd0-50f41a99447a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ecedd70b8d043668a6b7b01cbd6ee85, user_id=27c5678aa26644629197b10a46518d07, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:27.187724 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef69b7d3-374b-450a-8fd0-50f41a99447a admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:27.202015 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 892/1792] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:14:27 2022] POST /identity/v3/users => generated 300 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:14:27.204835 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cdaf32-4658-429b-aa67-8664a5a3a1a0 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:27.239867 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cdaf32-4658-429b-aa67-8664a5a3a1a0 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:27.242059 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 901/1793] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:14:26 2022] PUT /identity/v3/projects/0ecedd70b8d043668a6b7b01cbd6ee85/users/27c5678aa26644629197b10a46518d07/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:27.244384 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4809af6-87be-4293-947e-36191413c0c5 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:27.244676 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4809af6-87be-4293-947e-36191413c0c5 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:27.244954 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4809af6-87be-4293-947e-36191413c0c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:27.324042 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-a4809af6-87be-4293-947e-36191413c0c5 None None] MFA Rules not processed for user `27c5678aa26644629197b10a46518d07`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:27.325305 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-a4809af6-87be-4293-947e-36191413c0c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:27.461932 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228db146-ea62-4e97-97c1-0c8db57aa657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:27.465366 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228db146-ea62-4e97-97c1-0c8db57aa657 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:27.465764 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228db146-ea62-4e97-97c1-0c8db57aa657 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:27.466150 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228db146-ea62-4e97-97c1-0c8db57aa657 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:27.468295 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228db146-ea62-4e97-97c1-0c8db57aa657 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:27.472973 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228db146-ea62-4e97-97c1-0c8db57aa657 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:27.567367 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 893/1794] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:14:27 2022] GET /identity/v3/roles => generated 1420 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:14:27.570412 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228db146-ea62-4e97-97c1-0c8db57aa657 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:27.593145 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 902/1795] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:27 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:27.632734 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228db146-ea62-4e97-97c1-0c8db57aa657 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:27.634766 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4809af6-87be-4293-947e-36191413c0c5 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:27.663987 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:27.664647 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e697d9f-316d-4a90-b34a-51fefd9f1f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:27.666994 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e697d9f-316d-4a90-b34a-51fefd9f1f6f admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:27.668528 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e697d9f-316d-4a90-b34a-51fefd9f1f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:27.668933 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e697d9f-316d-4a90-b34a-51fefd9f1f6f admin admin] PATH_INFO: `/v3/projects/be5c31650e5b4c2a821a58af2cf90c21/users/32db6c5c470941088436eec99780eac6/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:27.748453 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e697d9f-316d-4a90-b34a-51fefd9f1f6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=be5c31650e5b4c2a821a58af2cf90c21, user_id=32db6c5c470941088436eec99780eac6, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:27.753174 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e697d9f-316d-4a90-b34a-51fefd9f1f6f admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:27.814463 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 894/1796] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:14:27 2022] PUT /identity/v3/projects/be5c31650e5b4c2a821a58af2cf90c21/users/32db6c5c470941088436eec99780eac6/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:27.822027 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318f1e6e-c0e3-4b37-a004-c0ec4ec3d991 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:27.825364 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318f1e6e-c0e3-4b37-a004-c0ec4ec3d991 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:27.845523 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318f1e6e-c0e3-4b37-a004-c0ec4ec3d991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:27.878366 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-318f1e6e-c0e3-4b37-a004-c0ec4ec3d991 None None] MFA Rules not processed for user `b0972dbd5cff4610b239326425e6351c`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:27.887713 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-318f1e6e-c0e3-4b37-a004-c0ec4ec3d991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:27.938359 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4809af6-87be-4293-947e-36191413c0c5 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:28.200227 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 895/1797] 10.222.0.30 () {58 vars in 1015 bytes} [Fri Aug 26 12:14:27 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 385 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:28.201179 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318f1e6e-c0e3-4b37-a004-c0ec4ec3d991 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:28.251773 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537ef4b0-195b-4fe4-b869-94f85d6a08d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:28.252624 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ef4b0-195b-4fe4-b869-94f85d6a08d9 service cinder] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:28.252871 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ef4b0-195b-4fe4-b869-94f85d6a08d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:28.253116 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ef4b0-195b-4fe4-b869-94f85d6a08d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:28.443640 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:28.465149 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318f1e6e-c0e3-4b37-a004-c0ec4ec3d991 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:28.526770 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537ef4b0-195b-4fe4-b869-94f85d6a08d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:28.530222 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537ef4b0-195b-4fe4-b869-94f85d6a08d9 service cinder] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:28.557796 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a31aa8b-c584-44dd-96dc-c2bc6e0cedbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:28.558449 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a31aa8b-c584-44dd-96dc-c2bc6e0cedbe admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:28.558738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a31aa8b-c584-44dd-96dc-c2bc6e0cedbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:28.562597 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a31aa8b-c584-44dd-96dc-c2bc6e0cedbe admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:28.574185 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a31aa8b-c584-44dd-96dc-c2bc6e0cedbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:28.577664 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 903/1798] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:14:27 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 944 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:14:28.593833 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a31aa8b-c584-44dd-96dc-c2bc6e0cedbe admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:28.681148 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 896/1799] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:14:28 2022] GET /identity/v3/roles => generated 1420 bytes in 480 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:14:28.685806 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537ef4b0-195b-4fe4-b869-94f85d6a08d9 service cinder] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:28.722001 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537ef4b0-195b-4fe4-b869-94f85d6a08d9 service cinder] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:28.781055 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d5c045-2834-4cc9-875b-306b1f6cdb6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:28.781055 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d5c045-2834-4cc9-875b-306b1f6cdb6b admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:28.781055 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d5c045-2834-4cc9-875b-306b1f6cdb6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:28.781055 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d5c045-2834-4cc9-875b-306b1f6cdb6b admin admin] PATH_INFO: `/v3/projects/be5c31650e5b4c2a821a58af2cf90c21/users/32db6c5c470941088436eec99780eac6/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:28.782724 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67054b4d-924b-4e95-90fd-485bb9c83d6e None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:28.783219 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67054b4d-924b-4e95-90fd-485bb9c83d6e None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:28.783736 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67054b4d-924b-4e95-90fd-485bb9c83d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:28.798303 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d5c045-2834-4cc9-875b-306b1f6cdb6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=be5c31650e5b4c2a821a58af2cf90c21, user_id=32db6c5c470941088436eec99780eac6, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:28.807042 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d5c045-2834-4cc9-875b-306b1f6cdb6b admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:28.832843 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-67054b4d-924b-4e95-90fd-485bb9c83d6e None None] MFA Rules not processed for user `36a27ca98a2e4521b269be48c9ec4cb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:28.838711 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-67054b4d-924b-4e95-90fd-485bb9c83d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:28.839167 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 904/1800] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:14:28 2022] PUT /identity/v3/projects/be5c31650e5b4c2a821a58af2cf90c21/users/32db6c5c470941088436eec99780eac6/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:28.851113 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c196e-c6a4-466e-9488-3cc1065d090d None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:28.852929 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c196e-c6a4-466e-9488-3cc1065d090d None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:28.854322 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c196e-c6a4-466e-9488-3cc1065d090d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:28.964734 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-247c196e-c6a4-466e-9488-3cc1065d090d None None] MFA Rules not processed for user `32db6c5c470941088436eec99780eac6`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:28.966360 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-247c196e-c6a4-466e-9488-3cc1065d090d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:29.033839 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 897/1801] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:29.037091 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67054b4d-924b-4e95-90fd-485bb9c83d6e None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:29.113238 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:29.222006 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 905/1802] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:28 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 373 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:29.277348 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67054b4d-924b-4e95-90fd-485bb9c83d6e None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:29.286510 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247c196e-c6a4-466e-9488-3cc1065d090d None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:29.288163 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-247c196e-c6a4-466e-9488-3cc1065d090d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:29.308184 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef73407e-5be4-4785-a756-fce9c4249561 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:29.312451 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73407e-5be4-4785-a756-fce9c4249561 service cinder] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:29.312771 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73407e-5be4-4785-a756-fce9c4249561 service cinder] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:29.313056 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73407e-5be4-4785-a756-fce9c4249561 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:29.330645 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247c196e-c6a4-466e-9488-3cc1065d090d None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:29.351111 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7066d98c-126d-45ec-a19c-2c407168ddec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:29.351679 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7066d98c-126d-45ec-a19c-2c407168ddec service cinder] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:29.351998 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7066d98c-126d-45ec-a19c-2c407168ddec service cinder] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:29.352278 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7066d98c-126d-45ec-a19c-2c407168ddec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:29.405343 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7066d98c-126d-45ec-a19c-2c407168ddec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:29.410795 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7066d98c-126d-45ec-a19c-2c407168ddec service cinder] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:29.459235 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 906/1803] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:14:29 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:14:29.502774 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef73407e-5be4-4785-a756-fce9c4249561 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:29.504957 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef73407e-5be4-4785-a756-fce9c4249561 service cinder] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:29.537549 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 898/1804] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:14:29 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:14:29.744804 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7066d98c-126d-45ec-a19c-2c407168ddec service cinder] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:29.746399 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-7066d98c-126d-45ec-a19c-2c407168ddec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:30.080339 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7066d98c-126d-45ec-a19c-2c407168ddec service cinder] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:30.101626 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a078afec-4e61-47f5-a0c5-758d6f98b630 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:30.103153 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078afec-4e61-47f5-a0c5-758d6f98b630 service neutron] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:30.103461 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078afec-4e61-47f5-a0c5-758d6f98b630 service neutron] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:30.105412 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078afec-4e61-47f5-a0c5-758d6f98b630 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:30.135982 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a078afec-4e61-47f5-a0c5-758d6f98b630 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:30.143988 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a078afec-4e61-47f5-a0c5-758d6f98b630 service neutron] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:30.194814 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 907/1805] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:14:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:14:30.578263 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef73407e-5be4-4785-a756-fce9c4249561 service cinder] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:30.726033 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:30.785057 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef73407e-5be4-4785-a756-fce9c4249561 service cinder] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:30.799349 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23fbdbf-e333-4cc1-b74d-f9da9604e16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:30.800028 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23fbdbf-e333-4cc1-b74d-f9da9604e16d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:30.800314 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23fbdbf-e333-4cc1-b74d-f9da9604e16d admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:30.800595 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23fbdbf-e333-4cc1-b74d-f9da9604e16d admin admin] PATH_INFO: `/v3/users/97b4d37ce3354f1d907607fb0a1de35e` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:30.805707 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23fbdbf-e333-4cc1-b74d-f9da9604e16d admin admin] RBAC: Authorizing `identity:delete_user(user_id=97b4d37ce3354f1d907607fb0a1de35e)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:30.808004 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23fbdbf-e333-4cc1-b74d-f9da9604e16d admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:31.017446 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-b23fbdbf-e333-4cc1-b74d-f9da9604e16d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97b4d37ce3354f1d907607fb0a1de35e'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:31.036729 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-b23fbdbf-e333-4cc1-b74d-f9da9604e16d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97b4d37ce3354f1d907607fb0a1de35e'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:31.053570 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-b23fbdbf-e333-4cc1-b74d-f9da9604e16d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97b4d37ce3354f1d907607fb0a1de35e'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:31.053570 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-b23fbdbf-e333-4cc1-b74d-f9da9604e16d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97b4d37ce3354f1d907607fb0a1de35e'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:31.101503 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-b23fbdbf-e333-4cc1-b74d-f9da9604e16d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97b4d37ce3354f1d907607fb0a1de35e'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:31.110631 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 899/1806] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:14:30 2022] DELETE /identity/v3/users/97b4d37ce3354f1d907607fb0a1de35e => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:31.116256 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1836c348-4221-422c-b73c-45c949971cf7 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:31.116786 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1836c348-4221-422c-b73c-45c949971cf7 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:31.116909 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1836c348-4221-422c-b73c-45c949971cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:31.182901 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-1836c348-4221-422c-b73c-45c949971cf7 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:31.184643 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-1836c348-4221-422c-b73c-45c949971cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:31.402844 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 908/1807] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:14:31.572052 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23fbdbf-e333-4cc1-b74d-f9da9604e16d admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:31.573930 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-b23fbdbf-e333-4cc1-b74d-f9da9604e16d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:31.786762 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23fbdbf-e333-4cc1-b74d-f9da9604e16d admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:31.832044 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38e1eca-1f38-4941-8c40-cd679c6e4e22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:31.840089 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e1eca-1f38-4941-8c40-cd679c6e4e22 service neutron] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:31.840705 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e1eca-1f38-4941-8c40-cd679c6e4e22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:31.841604 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e1eca-1f38-4941-8c40-cd679c6e4e22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:31.872499 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38e1eca-1f38-4941-8c40-cd679c6e4e22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:31.874594 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38e1eca-1f38-4941-8c40-cd679c6e4e22 service neutron] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:31.911263 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 900/1808] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:14:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:14:34.657418 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1836c348-4221-422c-b73c-45c949971cf7 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:34.663562 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-1836c348-4221-422c-b73c-45c949971cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:34.729511 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1836c348-4221-422c-b73c-45c949971cf7 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:34.780176 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf742d3-40fb-423d-aba2-287f2fc51e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:34.781020 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf742d3-40fb-423d-aba2-287f2fc51e57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:34.784717 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf742d3-40fb-423d-aba2-287f2fc51e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:34.788417 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf742d3-40fb-423d-aba2-287f2fc51e57 admin admin] PATH_INFO: `/v3/users/534019974dea45ca8b6a35bff319c5d0` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:34.791915 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf742d3-40fb-423d-aba2-287f2fc51e57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=534019974dea45ca8b6a35bff319c5d0)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:34.798575 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf742d3-40fb-423d-aba2-287f2fc51e57 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:35.035002 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-adf742d3-40fb-423d-aba2-287f2fc51e57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '534019974dea45ca8b6a35bff319c5d0'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:35.053463 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-adf742d3-40fb-423d-aba2-287f2fc51e57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '534019974dea45ca8b6a35bff319c5d0'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:35.054242 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-adf742d3-40fb-423d-aba2-287f2fc51e57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '534019974dea45ca8b6a35bff319c5d0'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:35.069432 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-adf742d3-40fb-423d-aba2-287f2fc51e57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '534019974dea45ca8b6a35bff319c5d0'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:35.071691 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-adf742d3-40fb-423d-aba2-287f2fc51e57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '534019974dea45ca8b6a35bff319c5d0'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:35.107544 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:35.158504 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 909/1809] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:14:34 2022] DELETE /identity/v3/users/534019974dea45ca8b6a35bff319c5d0 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:35.163553 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d2bffd-5844-4132-b532-c9b2d259c98f None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:35.164520 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d2bffd-5844-4132-b532-c9b2d259c98f None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:35.165015 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d2bffd-5844-4132-b532-c9b2d259c98f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:35.205872 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-32d2bffd-5844-4132-b532-c9b2d259c98f None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:35.207771 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-32d2bffd-5844-4132-b532-c9b2d259c98f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:35.350457 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 901/1810] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:35.378865 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf742d3-40fb-423d-aba2-287f2fc51e57 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:35.380634 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-adf742d3-40fb-423d-aba2-287f2fc51e57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:35.515173 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf742d3-40fb-423d-aba2-287f2fc51e57 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:35.519644 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d2bffd-5844-4132-b532-c9b2d259c98f None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:35.521411 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-32d2bffd-5844-4132-b532-c9b2d259c98f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:35.528590 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d003f909-80ca-401c-8733-b5b12edcdb4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:35.529747 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d003f909-80ca-401c-8733-b5b12edcdb4f service neutron] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:35.530314 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d003f909-80ca-401c-8733-b5b12edcdb4f service neutron] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:35.530734 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d003f909-80ca-401c-8733-b5b12edcdb4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:35.542009 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d2bffd-5844-4132-b532-c9b2d259c98f None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:35.547364 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb9eed8-3c10-4305-b456-307313f15419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:35.548212 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb9eed8-3c10-4305-b456-307313f15419 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:35.548693 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb9eed8-3c10-4305-b456-307313f15419 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:35.549138 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb9eed8-3c10-4305-b456-307313f15419 admin admin] PATH_INFO: `/v3/projects/be9334cf7b3a45939604edc1c9f2eeef` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:35.550563 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb9eed8-3c10-4305-b456-307313f15419 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be9334cf7b3a45939604edc1c9f2eeef)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:35.553354 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d003f909-80ca-401c-8733-b5b12edcdb4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:35.554190 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb9eed8-3c10-4305-b456-307313f15419 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:35.569558 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:35.573947 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d003f909-80ca-401c-8733-b5b12edcdb4f service neutron] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:35.596630 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 910/1811] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:14:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:14:35.650026 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-3fb9eed8-3c10-4305-b456-307313f15419 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be9334cf7b3a45939604edc1c9f2eeef', 'request_id': 'req-3fb9eed8-3c10-4305-b456-307313f15419'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:35.658309 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-3fb9eed8-3c10-4305-b456-307313f15419 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be9334cf7b3a45939604edc1c9f2eeef', 'request_id': 'req-3fb9eed8-3c10-4305-b456-307313f15419'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:35.668395 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-3fb9eed8-3c10-4305-b456-307313f15419 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be9334cf7b3a45939604edc1c9f2eeef', 'request_id': 'req-3fb9eed8-3c10-4305-b456-307313f15419'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:35.681131 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-3fb9eed8-3c10-4305-b456-307313f15419 admin admin] The token cache is being invalidate because project be9334cf7b3a45939604edc1c9f2eeef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:14:35.681823 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-3fb9eed8-3c10-4305-b456-307313f15419 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:35.682507 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-3fb9eed8-3c10-4305-b456-307313f15419 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:35.687628 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 902/1812] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:14:35 2022] DELETE /identity/v3/projects/be9334cf7b3a45939604edc1c9f2eeef => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:35.689496 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d003f909-80ca-401c-8733-b5b12edcdb4f service neutron] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:35.702618 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d003f909-80ca-401c-8733-b5b12edcdb4f service neutron] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:35.778346 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9054d12-bd4e-4622-ad6f-a5fbbae54fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:35.779014 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9054d12-bd4e-4622-ad6f-a5fbbae54fde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:35.779314 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9054d12-bd4e-4622-ad6f-a5fbbae54fde admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:35.779586 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9054d12-bd4e-4622-ad6f-a5fbbae54fde admin admin] PATH_INFO: `/v3/users/32db6c5c470941088436eec99780eac6` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:35.780859 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9054d12-bd4e-4622-ad6f-a5fbbae54fde admin admin] RBAC: Authorizing `identity:delete_user(user_id=32db6c5c470941088436eec99780eac6)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:35.782777 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9054d12-bd4e-4622-ad6f-a5fbbae54fde admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:35.829066 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad2fb1b-16cd-4367-a348-0c733f8ccfc7 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:35.830657 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad2fb1b-16cd-4367-a348-0c733f8ccfc7 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:35.831019 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad2fb1b-16cd-4367-a348-0c733f8ccfc7 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:35.834105 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 903/1813] 10.222.0.61 () {54 vars in 921 bytes} [Fri Aug 26 12:14:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:14:35.844504 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-d9054d12-bd4e-4622-ad6f-a5fbbae54fde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32db6c5c470941088436eec99780eac6'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:35.854131 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-d9054d12-bd4e-4622-ad6f-a5fbbae54fde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32db6c5c470941088436eec99780eac6'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:35.854914 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-d9054d12-bd4e-4622-ad6f-a5fbbae54fde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32db6c5c470941088436eec99780eac6'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:35.862816 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-d9054d12-bd4e-4622-ad6f-a5fbbae54fde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32db6c5c470941088436eec99780eac6'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:35.863828 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-d9054d12-bd4e-4622-ad6f-a5fbbae54fde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32db6c5c470941088436eec99780eac6'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:35.882471 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a418de-3019-4ed5-93e0-b175afc4937d None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:35.883409 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 911/1814] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:14:35 2022] DELETE /identity/v3/users/32db6c5c470941088436eec99780eac6 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 12:14:35.883804 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a418de-3019-4ed5-93e0-b175afc4937d None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:35.884185 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a418de-3019-4ed5-93e0-b175afc4937d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:35.906797 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-22a418de-3019-4ed5-93e0-b175afc4937d None None] MFA Rules not processed for user `7aacd7d6d17e4f10ba5c9107a12beafa`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:35.907916 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-22a418de-3019-4ed5-93e0-b175afc4937d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:36.056076 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 904/1815] 10.222.0.61 () {58 vars in 1018 bytes} [Fri Aug 26 12:14:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:36.129790 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9054d12-bd4e-4622-ad6f-a5fbbae54fde admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:36.131250 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-d9054d12-bd4e-4622-ad6f-a5fbbae54fde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:36.159519 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9054d12-bd4e-4622-ad6f-a5fbbae54fde admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:36.167693 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e333b7cf-805f-4cf6-b541-932ff1f99d57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aacd7d6d17e4f10ba5c9107a12beafa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:36.168687 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333b7cf-805f-4cf6-b541-932ff1f99d57 service nova] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:36.169218 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333b7cf-805f-4cf6-b541-932ff1f99d57 service nova] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:36.169674 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e333b7cf-805f-4cf6-b541-932ff1f99d57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:36.186425 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e333b7cf-805f-4cf6-b541-932ff1f99d57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:36.189269 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e333b7cf-805f-4cf6-b541-932ff1f99d57 service nova] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:36.196472 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:36.205601 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 912/1816] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:14:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:14:36.322456 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a418de-3019-4ed5-93e0-b175afc4937d None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:36.324514 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-22a418de-3019-4ed5-93e0-b175afc4937d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:36.439137 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a418de-3019-4ed5-93e0-b175afc4937d None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:36.443684 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51a5662-902d-4276-9c0a-8958c957d4e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:36.444317 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a5662-902d-4276-9c0a-8958c957d4e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:36.444594 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a5662-902d-4276-9c0a-8958c957d4e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:36.444865 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a5662-902d-4276-9c0a-8958c957d4e4 admin admin] PATH_INFO: `/v3/projects/04fb0d30ff60483fa2c9be44753b59ca` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:36.446186 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51a5662-902d-4276-9c0a-8958c957d4e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04fb0d30ff60483fa2c9be44753b59ca)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:36.448928 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51a5662-902d-4276-9c0a-8958c957d4e4 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:36.474627 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e333b7cf-805f-4cf6-b541-932ff1f99d57 service nova] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:36.477423 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-e333b7cf-805f-4cf6-b541-932ff1f99d57 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:36.497963 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e333b7cf-805f-4cf6-b541-932ff1f99d57 service nova] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:36.506908 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:36.534853 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-d51a5662-902d-4276-9c0a-8958c957d4e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04fb0d30ff60483fa2c9be44753b59ca', 'request_id': 'req-d51a5662-902d-4276-9c0a-8958c957d4e4'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:36.545318 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-d51a5662-902d-4276-9c0a-8958c957d4e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04fb0d30ff60483fa2c9be44753b59ca', 'request_id': 'req-d51a5662-902d-4276-9c0a-8958c957d4e4'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:36.554253 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-d51a5662-902d-4276-9c0a-8958c957d4e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04fb0d30ff60483fa2c9be44753b59ca', 'request_id': 'req-d51a5662-902d-4276-9c0a-8958c957d4e4'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:36.569082 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-d51a5662-902d-4276-9c0a-8958c957d4e4 admin admin] The token cache is being invalidate because project 04fb0d30ff60483fa2c9be44753b59ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:14:36.569470 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-d51a5662-902d-4276-9c0a-8958c957d4e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:36.570206 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-d51a5662-902d-4276-9c0a-8958c957d4e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:36.575253 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 905/1817] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:14:36 2022] DELETE /identity/v3/projects/04fb0d30ff60483fa2c9be44753b59ca => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:36.577463 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51a5662-902d-4276-9c0a-8958c957d4e4 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:36.579838 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-d51a5662-902d-4276-9c0a-8958c957d4e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:36.586712 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1f26f0-61b6-4157-aa1b-940e5be6988b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:36.587511 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1f26f0-61b6-4157-aa1b-940e5be6988b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:36.587892 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1f26f0-61b6-4157-aa1b-940e5be6988b admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:36.588264 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1f26f0-61b6-4157-aa1b-940e5be6988b admin admin] PATH_INFO: `/v3/projects/be5c31650e5b4c2a821a58af2cf90c21` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:36.589498 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1f26f0-61b6-4157-aa1b-940e5be6988b admin admin] RBAC: Authorizing `identity:delete_project(project_id=be5c31650e5b4c2a821a58af2cf90c21)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:36.591553 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1f26f0-61b6-4157-aa1b-940e5be6988b admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:36.595967 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51a5662-902d-4276-9c0a-8958c957d4e4 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:36.604682 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43af400c-54c8-4242-a0a3-908100c7866b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:36.605343 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43af400c-54c8-4242-a0a3-908100c7866b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:36.605744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43af400c-54c8-4242-a0a3-908100c7866b admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:36.605908 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43af400c-54c8-4242-a0a3-908100c7866b admin admin] PATH_INFO: `/v3/users/c02fb133772c407593d649e82b4252a3` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:36.607120 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43af400c-54c8-4242-a0a3-908100c7866b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c02fb133772c407593d649e82b4252a3)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:36.609601 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43af400c-54c8-4242-a0a3-908100c7866b admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:36.645739 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-8b1f26f0-61b6-4157-aa1b-940e5be6988b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be5c31650e5b4c2a821a58af2cf90c21', 'request_id': 'req-8b1f26f0-61b6-4157-aa1b-940e5be6988b'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:36.655561 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-8b1f26f0-61b6-4157-aa1b-940e5be6988b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be5c31650e5b4c2a821a58af2cf90c21', 'request_id': 'req-8b1f26f0-61b6-4157-aa1b-940e5be6988b'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:36.666773 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-8b1f26f0-61b6-4157-aa1b-940e5be6988b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be5c31650e5b4c2a821a58af2cf90c21', 'request_id': 'req-8b1f26f0-61b6-4157-aa1b-940e5be6988b'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:36.680884 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-8b1f26f0-61b6-4157-aa1b-940e5be6988b admin admin] The token cache is being invalidate because project be5c31650e5b4c2a821a58af2cf90c21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81712) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:14:36.681298 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-8b1f26f0-61b6-4157-aa1b-940e5be6988b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:36.682107 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-8b1f26f0-61b6-4157-aa1b-940e5be6988b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:36.683749 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 913/1818] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:14:36 2022] DELETE /identity/v3/projects/be5c31650e5b4c2a821a58af2cf90c21 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:36.688331 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f235119-7888-42ae-900a-75e224146271 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:36.688679 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f235119-7888-42ae-900a-75e224146271 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:36.688961 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f235119-7888-42ae-900a-75e224146271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:36.691581 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-43af400c-54c8-4242-a0a3-908100c7866b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c02fb133772c407593d649e82b4252a3'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:36.700236 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-43af400c-54c8-4242-a0a3-908100c7866b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c02fb133772c407593d649e82b4252a3'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:36.707728 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-43af400c-54c8-4242-a0a3-908100c7866b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c02fb133772c407593d649e82b4252a3'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:36.708402 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-43af400c-54c8-4242-a0a3-908100c7866b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c02fb133772c407593d649e82b4252a3'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:36.709644 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-4f235119-7888-42ae-900a-75e224146271 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:36.710852 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-4f235119-7888-42ae-900a-75e224146271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:36.721397 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-43af400c-54c8-4242-a0a3-908100c7866b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c02fb133772c407593d649e82b4252a3'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:36.727801 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 906/1819] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:14:36 2022] DELETE /identity/v3/users/c02fb133772c407593d649e82b4252a3 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 12:14:36.792948 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 914/1820] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:14:36.800326 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b90f96-ca86-4d67-8c35-7bdb1b4cbecd None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:36.800921 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b90f96-ca86-4d67-8c35-7bdb1b4cbecd None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:36.801464 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b90f96-ca86-4d67-8c35-7bdb1b4cbecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:36.829796 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-34b90f96-ca86-4d67-8c35-7bdb1b4cbecd None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:36.831722 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-34b90f96-ca86-4d67-8c35-7bdb1b4cbecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:36.909575 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 907/1821] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:36.913561 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f235119-7888-42ae-900a-75e224146271 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:36.925029 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f235119-7888-42ae-900a-75e224146271 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:36.930028 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3633dd-ab80-41ee-bd98-2ad0da06c4bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:36.930636 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3633dd-ab80-41ee-bd98-2ad0da06c4bc admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:36.930919 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3633dd-ab80-41ee-bd98-2ad0da06c4bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:36.931221 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3633dd-ab80-41ee-bd98-2ad0da06c4bc admin admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:36.931883 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3633dd-ab80-41ee-bd98-2ad0da06c4bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:14:36.932186 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3633dd-ab80-41ee-bd98-2ad0da06c4bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:36.933818 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3633dd-ab80-41ee-bd98-2ad0da06c4bc admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:36.946597 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 915/1822] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:14:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:14:36.951010 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b90f96-ca86-4d67-8c35-7bdb1b4cbecd None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:36.964471 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b90f96-ca86-4d67-8c35-7bdb1b4cbecd None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:36.972963 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0817ba-9b4a-47b6-a3ed-6d832de9b63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:36.973729 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0817ba-9b4a-47b6-a3ed-6d832de9b63c admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:36.974170 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0817ba-9b4a-47b6-a3ed-6d832de9b63c admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:36.974508 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0817ba-9b4a-47b6-a3ed-6d832de9b63c admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:36.975159 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0817ba-9b4a-47b6-a3ed-6d832de9b63c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:36.978915 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0817ba-9b4a-47b6-a3ed-6d832de9b63c admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:37.024589 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 908/1823] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:14:36 2022] POST /identity/v3/projects => generated 407 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 12:14:37.031788 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3633dd-ab80-41ee-bd98-2ad0da06c4bc admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:37.047077 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3633dd-ab80-41ee-bd98-2ad0da06c4bc admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:37.068706 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:37.078417 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0817ba-9b4a-47b6-a3ed-6d832de9b63c admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:37.080426 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-8e0817ba-9b4a-47b6-a3ed-6d832de9b63c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:37.100398 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=82,keystone:DELETE=15,keystone:INSERT=3 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:37.181950 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17737f54-c575-495a-990e-025de3ee05fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:37.181950 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17737f54-c575-495a-990e-025de3ee05fd admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:37.181950 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17737f54-c575-495a-990e-025de3ee05fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:37.181950 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17737f54-c575-495a-990e-025de3ee05fd admin admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:37.182689 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17737f54-c575-495a-990e-025de3ee05fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:37.182689 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17737f54-c575-495a-990e-025de3ee05fd admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:37.194715 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0817ba-9b4a-47b6-a3ed-6d832de9b63c admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:37.202465 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f997ee-a8e1-41de-a7a3-61ccf20dd2a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:37.203448 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f997ee-a8e1-41de-a7a3-61ccf20dd2a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:37.203751 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f997ee-a8e1-41de-a7a3-61ccf20dd2a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:37.204610 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f997ee-a8e1-41de-a7a3-61ccf20dd2a3 admin admin] PATH_INFO: `/v3/projects/bd793d6416e246cc90bd4ebc35a235e1` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:37.206799 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f997ee-a8e1-41de-a7a3-61ccf20dd2a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd793d6416e246cc90bd4ebc35a235e1)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:37.208931 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f997ee-a8e1-41de-a7a3-61ccf20dd2a3 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:37.215144 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 916/1824] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:14:37 2022] POST /identity/v3/users => generated 304 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:14:37.219052 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17737f54-c575-495a-990e-025de3ee05fd admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:37.234505 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17737f54-c575-495a-990e-025de3ee05fd admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:37.240443 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d77f037-96b7-46e5-98fb-c9f3aef4d924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:37.241210 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77f037-96b7-46e5-98fb-c9f3aef4d924 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:37.241508 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77f037-96b7-46e5-98fb-c9f3aef4d924 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:37.241789 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77f037-96b7-46e5-98fb-c9f3aef4d924 admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:37.242918 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d77f037-96b7-46e5-98fb-c9f3aef4d924 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:37.245087 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d77f037-96b7-46e5-98fb-c9f3aef4d924 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:37.272856 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 917/1825] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:14:37.275493 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d77f037-96b7-46e5-98fb-c9f3aef4d924 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:37.314086 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d77f037-96b7-46e5-98fb-c9f3aef4d924 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:37.328736 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-c4f997ee-a8e1-41de-a7a3-61ccf20dd2a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd793d6416e246cc90bd4ebc35a235e1', 'request_id': 'req-c4f997ee-a8e1-41de-a7a3-61ccf20dd2a3'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:37.353785 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-c4f997ee-a8e1-41de-a7a3-61ccf20dd2a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd793d6416e246cc90bd4ebc35a235e1', 'request_id': 'req-c4f997ee-a8e1-41de-a7a3-61ccf20dd2a3'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:37.382006 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-c4f997ee-a8e1-41de-a7a3-61ccf20dd2a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd793d6416e246cc90bd4ebc35a235e1', 'request_id': 'req-c4f997ee-a8e1-41de-a7a3-61ccf20dd2a3'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:37.398242 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-c4f997ee-a8e1-41de-a7a3-61ccf20dd2a3 admin admin] The token cache is being invalidate because project bd793d6416e246cc90bd4ebc35a235e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:14:37.399096 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-c4f997ee-a8e1-41de-a7a3-61ccf20dd2a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:37.404091 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-c4f997ee-a8e1-41de-a7a3-61ccf20dd2a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:37.406326 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 909/1826] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:14:37 2022] DELETE /identity/v3/projects/bd793d6416e246cc90bd4ebc35a235e1 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:37.422260 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4894b925-605c-4a64-8313-639965119c60 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:37.422568 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4894b925-605c-4a64-8313-639965119c60 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:37.423746 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4894b925-605c-4a64-8313-639965119c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:37.428101 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356473ae-e11d-48ac-8382-a5ed7ddf1db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:37.429074 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356473ae-e11d-48ac-8382-a5ed7ddf1db5 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:37.429549 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356473ae-e11d-48ac-8382-a5ed7ddf1db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:37.430065 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356473ae-e11d-48ac-8382-a5ed7ddf1db5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:37.430855 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356473ae-e11d-48ac-8382-a5ed7ddf1db5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:37.432684 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356473ae-e11d-48ac-8382-a5ed7ddf1db5 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:37.457126 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 918/1827] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:14:37.458040 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356473ae-e11d-48ac-8382-a5ed7ddf1db5 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:37.460123 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-356473ae-e11d-48ac-8382-a5ed7ddf1db5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:37.475707 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356473ae-e11d-48ac-8382-a5ed7ddf1db5 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:37.483733 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a528af-23ba-4a15-91ca-dffbc60a74ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:37.484527 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a528af-23ba-4a15-91ca-dffbc60a74ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:37.485518 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a528af-23ba-4a15-91ca-dffbc60a74ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:37.485890 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a528af-23ba-4a15-91ca-dffbc60a74ff admin admin] PATH_INFO: `/v3/projects/5a903596b7d247648c39ebc48695ac86/users/e5e0d43e61894d1c9353178a128190e7/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:37.486237 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-4894b925-605c-4a64-8313-639965119c60 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:37.511281 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-4894b925-605c-4a64-8313-639965119c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:37.512116 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a528af-23ba-4a15-91ca-dffbc60a74ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a903596b7d247648c39ebc48695ac86, user_id=e5e0d43e61894d1c9353178a128190e7, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:37.514969 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a528af-23ba-4a15-91ca-dffbc60a74ff admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:37.529009 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 910/1828] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:14:37.542276 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64f5e69-ea37-4364-b267-ea1987566dcc None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:37.547763 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64f5e69-ea37-4364-b267-ea1987566dcc None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:37.553597 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64f5e69-ea37-4364-b267-ea1987566dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:37.558086 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 919/1829] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:14:37 2022] PUT /identity/v3/projects/5a903596b7d247648c39ebc48695ac86/users/e5e0d43e61894d1c9353178a128190e7/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:37.575151 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c6d5f6-16e4-4d25-a971-b71128f8edd4 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:37.579521 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c6d5f6-16e4-4d25-a971-b71128f8edd4 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:37.579948 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c6d5f6-16e4-4d25-a971-b71128f8edd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:37.585811 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-d64f5e69-ea37-4364-b267-ea1987566dcc None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:37.587468 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-d64f5e69-ea37-4364-b267-ea1987566dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:37.665327 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-75c6d5f6-16e4-4d25-a971-b71128f8edd4 None None] MFA Rules not processed for user `e5e0d43e61894d1c9353178a128190e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:37.666956 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-75c6d5f6-16e4-4d25-a971-b71128f8edd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:37.725700 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 911/1830] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:37.731530 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64f5e69-ea37-4364-b267-ea1987566dcc None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:37.760079 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64f5e69-ea37-4364-b267-ea1987566dcc None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:37.782138 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d96d01-8a19-4323-9c93-fe58d76b1fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:37.782761 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d96d01-8a19-4323-9c93-fe58d76b1fb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:37.783036 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d96d01-8a19-4323-9c93-fe58d76b1fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:37.783338 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d96d01-8a19-4323-9c93-fe58d76b1fb2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:37.783985 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d96d01-8a19-4323-9c93-fe58d76b1fb2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:14:37.784285 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d96d01-8a19-4323-9c93-fe58d76b1fb2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:37.785893 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d96d01-8a19-4323-9c93-fe58d76b1fb2 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:37.825378 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 912/1831] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:14:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:14:37.829749 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d96d01-8a19-4323-9c93-fe58d76b1fb2 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:37.844513 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d96d01-8a19-4323-9c93-fe58d76b1fb2 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:37.859516 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fca7ebb-0b46-497b-b381-27429db5eac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:37.859702 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fca7ebb-0b46-497b-b381-27429db5eac1 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:37.860509 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fca7ebb-0b46-497b-b381-27429db5eac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:37.863470 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fca7ebb-0b46-497b-b381-27429db5eac1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:37.863470 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fca7ebb-0b46-497b-b381-27429db5eac1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:37.863780 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fca7ebb-0b46-497b-b381-27429db5eac1 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:37.918351 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:37.940468 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 913/1832] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:14:37 2022] POST /identity/v3/projects => generated 375 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:14:37.940769 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:37.942243 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fca7ebb-0b46-497b-b381-27429db5eac1 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:37.971845 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 920/1833] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:37 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 398 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:37.976210 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c6d5f6-16e4-4d25-a971-b71128f8edd4 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:37.999360 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c6d5f6-16e4-4d25-a971-b71128f8edd4 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:38.110685 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fca7ebb-0b46-497b-b381-27429db5eac1 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:38.136020 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ff73aa-e802-4ffa-9ee7-faf4b9fb6e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:38.136660 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff73aa-e802-4ffa-9ee7-faf4b9fb6e74 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:38.137118 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff73aa-e802-4ffa-9ee7-faf4b9fb6e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:38.137398 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff73aa-e802-4ffa-9ee7-faf4b9fb6e74 admin admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:38.138789 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ff73aa-e802-4ffa-9ee7-faf4b9fb6e74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:38.141068 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ff73aa-e802-4ffa-9ee7-faf4b9fb6e74 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:38.164086 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35840655-2251-4371-b138-5d277cf47908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:38.164746 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35840655-2251-4371-b138-5d277cf47908 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:38.165030 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35840655-2251-4371-b138-5d277cf47908 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:38.165307 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35840655-2251-4371-b138-5d277cf47908 admin admin] PATH_INFO: `/v3/projects` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:38.165988 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35840655-2251-4371-b138-5d277cf47908 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:38.172664 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35840655-2251-4371-b138-5d277cf47908 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:38.216049 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 914/1834] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:14:37 2022] POST /identity/v3/users => generated 288 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:14:38.219225 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ff73aa-e802-4ffa-9ee7-faf4b9fb6e74 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:38.243566 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 921/1835] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:14:37 2022] POST /identity/v3/projects => generated 407 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:14:38.246883 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ff73aa-e802-4ffa-9ee7-faf4b9fb6e74 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:38.248149 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35840655-2251-4371-b138-5d277cf47908 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:38.407699 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7159b9-1f6b-4554-b1ab-14b2109f030e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:38.408435 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7159b9-1f6b-4554-b1ab-14b2109f030e admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:38.408799 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7159b9-1f6b-4554-b1ab-14b2109f030e admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:38.409079 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7159b9-1f6b-4554-b1ab-14b2109f030e admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:38.409775 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7159b9-1f6b-4554-b1ab-14b2109f030e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:38.412852 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7159b9-1f6b-4554-b1ab-14b2109f030e admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:38.424788 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35840655-2251-4371-b138-5d277cf47908 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:38.440055 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5654ef4c-1222-4cb3-a1f7-1e1a91a71fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:38.441003 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5654ef4c-1222-4cb3-a1f7-1e1a91a71fbc admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:38.441292 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5654ef4c-1222-4cb3-a1f7-1e1a91a71fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:38.441755 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5654ef4c-1222-4cb3-a1f7-1e1a91a71fbc admin admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:38.442404 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5654ef4c-1222-4cb3-a1f7-1e1a91a71fbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:38.443089 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 915/1836] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:14:38.444887 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7159b9-1f6b-4554-b1ab-14b2109f030e admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:38.447310 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5654ef4c-1222-4cb3-a1f7-1e1a91a71fbc admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:38.469532 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7159b9-1f6b-4554-b1ab-14b2109f030e admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:38.482509 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d0ce35-65f7-4beb-b3a6-5c60ea78e045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:38.483133 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d0ce35-65f7-4beb-b3a6-5c60ea78e045 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:38.483424 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d0ce35-65f7-4beb-b3a6-5c60ea78e045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:38.483602 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d0ce35-65f7-4beb-b3a6-5c60ea78e045 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:38.484266 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d0ce35-65f7-4beb-b3a6-5c60ea78e045 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:38.485847 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d0ce35-65f7-4beb-b3a6-5c60ea78e045 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:38.498747 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 922/1837] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:14:38 2022] POST /identity/v3/users => generated 310 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:14:38.500162 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5654ef4c-1222-4cb3-a1f7-1e1a91a71fbc admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:38.523092 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5654ef4c-1222-4cb3-a1f7-1e1a91a71fbc admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:38.524072 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 916/1838] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:14:38.526407 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d0ce35-65f7-4beb-b3a6-5c60ea78e045 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:38.553474 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcdba58-c763-4749-8558-d41d3ac012fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:38.554294 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcdba58-c763-4749-8558-d41d3ac012fb admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:38.554550 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcdba58-c763-4749-8558-d41d3ac012fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:38.554905 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcdba58-c763-4749-8558-d41d3ac012fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:38.555711 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcdba58-c763-4749-8558-d41d3ac012fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:38.561120 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcdba58-c763-4749-8558-d41d3ac012fb admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:38.565044 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d0ce35-65f7-4beb-b3a6-5c60ea78e045 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:38.579409 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354cfa3-c957-42c7-b78c-db429efb0928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:38.580257 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354cfa3-c957-42c7-b78c-db429efb0928 admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:38.580257 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354cfa3-c957-42c7-b78c-db429efb0928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:38.580640 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354cfa3-c957-42c7-b78c-db429efb0928 admin admin] PATH_INFO: `/v3/projects/5623f772dd7540a0971b12e54f9bdf49/users/b17b06a36ad645808128093781c7b409/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:38.595422 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 923/1839] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:14:38.599230 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcdba58-c763-4749-8558-d41d3ac012fb admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:38.608136 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3354cfa3-c957-42c7-b78c-db429efb0928 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5623f772dd7540a0971b12e54f9bdf49, user_id=b17b06a36ad645808128093781c7b409, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:38.611294 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3354cfa3-c957-42c7-b78c-db429efb0928 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:38.641056 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcdba58-c763-4749-8558-d41d3ac012fb admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:38.654067 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 917/1840] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:14:38 2022] PUT /identity/v3/projects/5623f772dd7540a0971b12e54f9bdf49/users/b17b06a36ad645808128093781c7b409/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 12:14:38.667937 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3159eac-8c45-4bc5-92f1-0b2a950bbc27 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:38.668232 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3159eac-8c45-4bc5-92f1-0b2a950bbc27 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:38.668735 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3159eac-8c45-4bc5-92f1-0b2a950bbc27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:38.835696 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-d3159eac-8c45-4bc5-92f1-0b2a950bbc27 None None] MFA Rules not processed for user `b17b06a36ad645808128093781c7b409`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:38.840934 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-d3159eac-8c45-4bc5-92f1-0b2a950bbc27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:38.880757 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3cedc3-9cbe-415d-8733-a71a4aaea093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:38.882241 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3cedc3-9cbe-415d-8733-a71a4aaea093 admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:38.882773 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3cedc3-9cbe-415d-8733-a71a4aaea093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:38.882879 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3cedc3-9cbe-415d-8733-a71a4aaea093 admin admin] PATH_INFO: `/v3/projects/bd63eb2f3f5c4752a3edc3c9b2e63c53/users/ff21489c72c649f68e30ea0ece1f2fca/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:38.891836 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:38.940150 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3cedc3-9cbe-415d-8733-a71a4aaea093 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd63eb2f3f5c4752a3edc3c9b2e63c53, user_id=ff21489c72c649f68e30ea0ece1f2fca, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:38.945597 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3cedc3-9cbe-415d-8733-a71a4aaea093 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:38.978762 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 924/1841] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:14:38 2022] PUT /identity/v3/projects/bd63eb2f3f5c4752a3edc3c9b2e63c53/users/ff21489c72c649f68e30ea0ece1f2fca/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:38.981341 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3cedc3-9cbe-415d-8733-a71a4aaea093 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:38.987510 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:39.119749 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 918/1842] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 453 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:39.121226 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3159eac-8c45-4bc5-92f1-0b2a950bbc27 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:39.181992 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3159eac-8c45-4bc5-92f1-0b2a950bbc27 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:39.313417 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3cedc3-9cbe-415d-8733-a71a4aaea093 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:39.334990 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114dc22a-45a8-4aef-ab88-a404fa40ef67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:39.338435 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114dc22a-45a8-4aef-ab88-a404fa40ef67 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:39.341278 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114dc22a-45a8-4aef-ab88-a404fa40ef67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:39.343135 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114dc22a-45a8-4aef-ab88-a404fa40ef67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:39.349618 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114dc22a-45a8-4aef-ab88-a404fa40ef67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:39.360791 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114dc22a-45a8-4aef-ab88-a404fa40ef67 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:39.377886 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a1b830-7364-4c52-8747-b0594df53535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:39.378614 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a1b830-7364-4c52-8747-b0594df53535 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:39.378905 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a1b830-7364-4c52-8747-b0594df53535 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:39.379326 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a1b830-7364-4c52-8747-b0594df53535 admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:39.380394 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a1b830-7364-4c52-8747-b0594df53535 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:39.385121 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a1b830-7364-4c52-8747-b0594df53535 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:39.401028 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 925/1843] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 420 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:14:39.405031 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114dc22a-45a8-4aef-ab88-a404fa40ef67 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:39.454155 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114dc22a-45a8-4aef-ab88-a404fa40ef67 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:39.466048 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 919/1844] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:14:39 2022] POST /identity/v3/projects => generated 375 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:14:39.467003 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a1b830-7364-4c52-8747-b0594df53535 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:39.627405 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a1b830-7364-4c52-8747-b0594df53535 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:39.640525 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86efbb02-1953-4d18-9780-ed645813e2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:39.643304 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86efbb02-1953-4d18-9780-ed645813e2e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:39.644707 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86efbb02-1953-4d18-9780-ed645813e2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:39.645428 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86efbb02-1953-4d18-9780-ed645813e2e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:39.648069 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86efbb02-1953-4d18-9780-ed645813e2e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:39.654579 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86efbb02-1953-4d18-9780-ed645813e2e5 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:39.734005 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 920/1845] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:14:39 2022] POST /identity/v3/users => generated 294 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:14:39.734926 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86efbb02-1953-4d18-9780-ed645813e2e5 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:39.794312 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86efbb02-1953-4d18-9780-ed645813e2e5 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:39.818124 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84c6f7e-8f3b-48f5-aedf-64e9639c69b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:39.818918 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c6f7e-8f3b-48f5-aedf-64e9639c69b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:39.819235 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c6f7e-8f3b-48f5-aedf-64e9639c69b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:39.819525 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c6f7e-8f3b-48f5-aedf-64e9639c69b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:39.831639 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84c6f7e-8f3b-48f5-aedf-64e9639c69b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:39.840113 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84c6f7e-8f3b-48f5-aedf-64e9639c69b2 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:39.879815 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 921/1846] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:14:39 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:14:39.884974 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84c6f7e-8f3b-48f5-aedf-64e9639c69b2 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:39.888414 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23cbdfa-c7a2-449d-aee4-6e9173da19cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:39.893047 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23cbdfa-c7a2-449d-aee4-6e9173da19cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:39.893349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23cbdfa-c7a2-449d-aee4-6e9173da19cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:39.893636 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23cbdfa-c7a2-449d-aee4-6e9173da19cb admin admin] PATH_INFO: `/v3/projects/bd63eb2f3f5c4752a3edc3c9b2e63c53/users/ff21489c72c649f68e30ea0ece1f2fca/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:39.900164 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84c6f7e-8f3b-48f5-aedf-64e9639c69b2 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:39.904095 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23cbdfa-c7a2-449d-aee4-6e9173da19cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd63eb2f3f5c4752a3edc3c9b2e63c53, user_id=ff21489c72c649f68e30ea0ece1f2fca, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:39.907027 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975486ab-1392-4492-b623-23858ef0b31e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:39.908400 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975486ab-1392-4492-b623-23858ef0b31e admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:39.908959 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975486ab-1392-4492-b623-23858ef0b31e admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:39.909526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975486ab-1392-4492-b623-23858ef0b31e admin admin] PATH_INFO: `/v3/projects/39eabc8708ea47119fa8fbb3f3306966/users/31003735cf8f4f7098561d3032e97363/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:39.920670 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23cbdfa-c7a2-449d-aee4-6e9173da19cb admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:39.937190 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975486ab-1392-4492-b623-23858ef0b31e admin admin] RBAC: Authorizing `identity:create_grant(project_id=39eabc8708ea47119fa8fbb3f3306966, user_id=31003735cf8f4f7098561d3032e97363, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:39.943732 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975486ab-1392-4492-b623-23858ef0b31e admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:39.960380 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 926/1847] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:14:39 2022] PUT /identity/v3/projects/bd63eb2f3f5c4752a3edc3c9b2e63c53/users/ff21489c72c649f68e30ea0ece1f2fca/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:39.963030 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389e2c72-0204-4b8a-bacb-3a4aa3dbbc80 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:39.963750 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389e2c72-0204-4b8a-bacb-3a4aa3dbbc80 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:39.964196 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389e2c72-0204-4b8a-bacb-3a4aa3dbbc80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:39.981699 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 922/1848] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:14:39 2022] PUT /identity/v3/projects/39eabc8708ea47119fa8fbb3f3306966/users/31003735cf8f4f7098561d3032e97363/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:39.985888 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975486ab-1392-4492-b623-23858ef0b31e admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:40.009132 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-389e2c72-0204-4b8a-bacb-3a4aa3dbbc80 None None] MFA Rules not processed for user `ff21489c72c649f68e30ea0ece1f2fca`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:40.012563 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-389e2c72-0204-4b8a-bacb-3a4aa3dbbc80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:40.045348 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:40.165679 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:40.175599 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 927/1849] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:39 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:40.181380 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975486ab-1392-4492-b623-23858ef0b31e admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:40.186527 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389e2c72-0204-4b8a-bacb-3a4aa3dbbc80 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:40.188696 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-389e2c72-0204-4b8a-bacb-3a4aa3dbbc80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:40.195903 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6780e8a4-c76d-42b4-91c3-f65f67fa5c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:40.196639 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6780e8a4-c76d-42b4-91c3-f65f67fa5c2b admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:40.196925 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6780e8a4-c76d-42b4-91c3-f65f67fa5c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:40.197202 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6780e8a4-c76d-42b4-91c3-f65f67fa5c2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:40.203448 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6780e8a4-c76d-42b4-91c3-f65f67fa5c2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:40.205670 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6780e8a4-c76d-42b4-91c3-f65f67fa5c2b admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:40.233333 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 923/1850] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:14:39 2022] GET /identity/v3/roles => generated 1420 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 12:14:40.238036 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6780e8a4-c76d-42b4-91c3-f65f67fa5c2b admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:40.280376 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6780e8a4-c76d-42b4-91c3-f65f67fa5c2b admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:40.302303 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e7ed93-67ae-45bb-b1e9-119400fcbfd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:40.303681 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7ed93-67ae-45bb-b1e9-119400fcbfd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:40.311091 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7ed93-67ae-45bb-b1e9-119400fcbfd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:40.312354 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7ed93-67ae-45bb-b1e9-119400fcbfd1 admin admin] PATH_INFO: `/v3/projects/39eabc8708ea47119fa8fbb3f3306966/users/31003735cf8f4f7098561d3032e97363/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:40.320040 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e7ed93-67ae-45bb-b1e9-119400fcbfd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39eabc8708ea47119fa8fbb3f3306966, user_id=31003735cf8f4f7098561d3032e97363, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:40.330608 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e7ed93-67ae-45bb-b1e9-119400fcbfd1 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:40.364252 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 924/1851] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:14:40 2022] PUT /identity/v3/projects/39eabc8708ea47119fa8fbb3f3306966/users/31003735cf8f4f7098561d3032e97363/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:40.380797 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a010dd-47c2-4dd9-9ede-f85e5a7ff9ef None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:40.380797 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a010dd-47c2-4dd9-9ede-f85e5a7ff9ef None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:40.381092 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a010dd-47c2-4dd9-9ede-f85e5a7ff9ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:40.421075 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389e2c72-0204-4b8a-bacb-3a4aa3dbbc80 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:40.506072 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-88a010dd-47c2-4dd9-9ede-f85e5a7ff9ef None None] MFA Rules not processed for user `31003735cf8f4f7098561d3032e97363`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:40.508776 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-88a010dd-47c2-4dd9-9ede-f85e5a7ff9ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:40.527973 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb07374-7419-4ebf-b25c-abd711c9bb1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:40.528758 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb07374-7419-4ebf-b25c-abd711c9bb1d service cinder] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:40.529042 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb07374-7419-4ebf-b25c-abd711c9bb1d service cinder] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:40.529312 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb07374-7419-4ebf-b25c-abd711c9bb1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:40.717732 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 925/1852] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:40 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:40.725854 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa00447e-624a-469e-aec2-ccc76ddb6220 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:40.726258 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa00447e-624a-469e-aec2-ccc76ddb6220 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:40.728121 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa00447e-624a-469e-aec2-ccc76ddb6220 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:40.730906 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 926/1853] 10.222.0.61 () {54 vars in 921 bytes} [Fri Aug 26 12:14:40 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:14:40.733297 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa00447e-624a-469e-aec2-ccc76ddb6220 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:40.740167 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-aa00447e-624a-469e-aec2-ccc76ddb6220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:40.785761 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb07374-7419-4ebf-b25c-abd711c9bb1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:40.790399 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb07374-7419-4ebf-b25c-abd711c9bb1d service cinder] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:40.816224 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 928/1854] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:14:40 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 630 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:14:40.817164 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb07374-7419-4ebf-b25c-abd711c9bb1d service cinder] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:40.820072 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-ddb07374-7419-4ebf-b25c-abd711c9bb1d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:40.853315 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb07374-7419-4ebf-b25c-abd711c9bb1d service cinder] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:40.860021 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d23eea-16bb-4cf0-a95a-342d74aa7a10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:40.860698 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d23eea-16bb-4cf0-a95a-342d74aa7a10 service cinder] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:40.860984 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d23eea-16bb-4cf0-a95a-342d74aa7a10 service cinder] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:40.861255 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d23eea-16bb-4cf0-a95a-342d74aa7a10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:40.984076 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d23eea-16bb-4cf0-a95a-342d74aa7a10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:40.984738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa00447e-624a-469e-aec2-ccc76ddb6220 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:40.985189 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d23eea-16bb-4cf0-a95a-342d74aa7a10 service cinder] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:40.997368 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86c6fa0-8509-4abf-b666-c8a1e36ac8eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:40.998165 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86c6fa0-8509-4abf-b666-c8a1e36ac8eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:40.998765 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86c6fa0-8509-4abf-b666-c8a1e36ac8eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:40.999001 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 929/1855] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:14:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:14:40.999400 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86c6fa0-8509-4abf-b666-c8a1e36ac8eb admin admin] PATH_INFO: `/v3/users/e918f83f201c47faaf8355111ce74914` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:41.000670 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86c6fa0-8509-4abf-b666-c8a1e36ac8eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=e918f83f201c47faaf8355111ce74914)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:41.002949 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86c6fa0-8509-4abf-b666-c8a1e36ac8eb admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:41.073717 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:41.139018 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-d86c6fa0-8509-4abf-b666-c8a1e36ac8eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e918f83f201c47faaf8355111ce74914'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:41.166139 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-d86c6fa0-8509-4abf-b666-c8a1e36ac8eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e918f83f201c47faaf8355111ce74914'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:41.196929 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-d86c6fa0-8509-4abf-b666-c8a1e36ac8eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e918f83f201c47faaf8355111ce74914'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:41.199362 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-d86c6fa0-8509-4abf-b666-c8a1e36ac8eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e918f83f201c47faaf8355111ce74914'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:41.251593 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-d86c6fa0-8509-4abf-b666-c8a1e36ac8eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e918f83f201c47faaf8355111ce74914'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:41.272402 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 927/1856] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:14:40 2022] DELETE /identity/v3/users/e918f83f201c47faaf8355111ce74914 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:41.277233 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270b329d-3fd0-4919-9a9b-bd18f7026f09 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:41.277522 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270b329d-3fd0-4919-9a9b-bd18f7026f09 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:41.277798 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270b329d-3fd0-4919-9a9b-bd18f7026f09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:41.295546 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:41.327436 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-270b329d-3fd0-4919-9a9b-bd18f7026f09 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:41.328907 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-270b329d-3fd0-4919-9a9b-bd18f7026f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:41.487586 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 930/1857] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:41.512713 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e152ef-9f04-4919-a322-ce9b691a9d1f None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:41.513496 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e152ef-9f04-4919-a322-ce9b691a9d1f None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:41.513496 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e152ef-9f04-4919-a322-ce9b691a9d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:41.538580 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-16e152ef-9f04-4919-a322-ce9b691a9d1f None None] MFA Rules not processed for user `7aacd7d6d17e4f10ba5c9107a12beafa`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:41.540752 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-16e152ef-9f04-4919-a322-ce9b691a9d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:41.596241 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270b329d-3fd0-4919-9a9b-bd18f7026f09 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:41.599186 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-270b329d-3fd0-4919-9a9b-bd18f7026f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:41.835286 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 928/1858] 10.222.0.61 () {58 vars in 1018 bytes} [Fri Aug 26 12:14:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:41.837568 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e152ef-9f04-4919-a322-ce9b691a9d1f None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:41.841118 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-16e152ef-9f04-4919-a322-ce9b691a9d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:41.864942 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270b329d-3fd0-4919-9a9b-bd18f7026f09 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:41.876063 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c71b81-8cff-4693-ba75-21b0f65d275f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:41.876871 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c71b81-8cff-4693-ba75-21b0f65d275f service neutron] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:41.877154 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c71b81-8cff-4693-ba75-21b0f65d275f service neutron] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:41.879367 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c71b81-8cff-4693-ba75-21b0f65d275f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:41.908968 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c71b81-8cff-4693-ba75-21b0f65d275f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:41.911185 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c71b81-8cff-4693-ba75-21b0f65d275f service neutron] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:41.939985 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 931/1859] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:14:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:14:41.983650 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e152ef-9f04-4919-a322-ce9b691a9d1f None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:41.990901 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c3bab9-4398-4ab2-820f-98ec48b32f0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:41.991631 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c3bab9-4398-4ab2-820f-98ec48b32f0f service cinder] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:41.991917 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c3bab9-4398-4ab2-820f-98ec48b32f0f service cinder] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:41.992964 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c3bab9-4398-4ab2-820f-98ec48b32f0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:41.995658 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-29c3bab9-4398-4ab2-820f-98ec48b32f0f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:42.077541 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:42.111022 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c3bab9-4398-4ab2-820f-98ec48b32f0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:42.112745 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c3bab9-4398-4ab2-820f-98ec48b32f0f service cinder] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:42.128251 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 929/1860] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:14:41 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:14:42.255442 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c71b81-8cff-4693-ba75-21b0f65d275f service neutron] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:42.259874 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-e2c71b81-8cff-4693-ba75-21b0f65d275f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:42.285506 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c71b81-8cff-4693-ba75-21b0f65d275f service neutron] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:42.295538 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56578e0-0050-4e36-bfd8-6bf99088b97e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aacd7d6d17e4f10ba5c9107a12beafa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:42.296315 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56578e0-0050-4e36-bfd8-6bf99088b97e service nova] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:42.296495 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56578e0-0050-4e36-bfd8-6bf99088b97e service nova] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:42.296814 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56578e0-0050-4e36-bfd8-6bf99088b97e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:42.312516 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56578e0-0050-4e36-bfd8-6bf99088b97e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:42.315481 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56578e0-0050-4e36-bfd8-6bf99088b97e service nova] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:42.331156 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 932/1861] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:14:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:14:42.908684 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c3bab9-4398-4ab2-820f-98ec48b32f0f service cinder] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:42.910495 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-29c3bab9-4398-4ab2-820f-98ec48b32f0f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:42.930812 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c3bab9-4398-4ab2-820f-98ec48b32f0f service cinder] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:42.937874 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22888b7-5a9e-479a-ac2e-ec6d6bc47d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:42.938753 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22888b7-5a9e-479a-ac2e-ec6d6bc47d8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:42.938860 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22888b7-5a9e-479a-ac2e-ec6d6bc47d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:42.939114 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22888b7-5a9e-479a-ac2e-ec6d6bc47d8d admin admin] PATH_INFO: `/v3/projects/064ad4d5e69d4ac4aaa35ddbeead2cb7` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:42.940211 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22888b7-5a9e-479a-ac2e-ec6d6bc47d8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=064ad4d5e69d4ac4aaa35ddbeead2cb7)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:42.950957 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22888b7-5a9e-479a-ac2e-ec6d6bc47d8d admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:43.005802 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-b22888b7-5a9e-479a-ac2e-ec6d6bc47d8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '064ad4d5e69d4ac4aaa35ddbeead2cb7', 'request_id': 'req-b22888b7-5a9e-479a-ac2e-ec6d6bc47d8d'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:43.013103 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-b22888b7-5a9e-479a-ac2e-ec6d6bc47d8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '064ad4d5e69d4ac4aaa35ddbeead2cb7', 'request_id': 'req-b22888b7-5a9e-479a-ac2e-ec6d6bc47d8d'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:43.023393 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-b22888b7-5a9e-479a-ac2e-ec6d6bc47d8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '064ad4d5e69d4ac4aaa35ddbeead2cb7', 'request_id': 'req-b22888b7-5a9e-479a-ac2e-ec6d6bc47d8d'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:43.041130 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-b22888b7-5a9e-479a-ac2e-ec6d6bc47d8d admin admin] The token cache is being invalidate because project 064ad4d5e69d4ac4aaa35ddbeead2cb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:14:43.041432 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-b22888b7-5a9e-479a-ac2e-ec6d6bc47d8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:43.042443 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-b22888b7-5a9e-479a-ac2e-ec6d6bc47d8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:43.044310 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 930/1862] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:14:42 2022] DELETE /identity/v3/projects/064ad4d5e69d4ac4aaa35ddbeead2cb7 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:43.048454 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56578e0-0050-4e36-bfd8-6bf99088b97e service nova] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:43.064358 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56578e0-0050-4e36-bfd8-6bf99088b97e service nova] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:43.140829 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13b6208-1468-4631-9776-06ef80c67829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:43.141605 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13b6208-1468-4631-9776-06ef80c67829 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:43.141891 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13b6208-1468-4631-9776-06ef80c67829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:43.142769 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13b6208-1468-4631-9776-06ef80c67829 admin admin] PATH_INFO: `/v3/users/27c5678aa26644629197b10a46518d07` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:43.144199 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13b6208-1468-4631-9776-06ef80c67829 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27c5678aa26644629197b10a46518d07)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:43.146590 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13b6208-1468-4631-9776-06ef80c67829 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:43.225110 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:43.239853 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-c13b6208-1468-4631-9776-06ef80c67829 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27c5678aa26644629197b10a46518d07'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:43.248538 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-c13b6208-1468-4631-9776-06ef80c67829 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27c5678aa26644629197b10a46518d07'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:43.249122 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-c13b6208-1468-4631-9776-06ef80c67829 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27c5678aa26644629197b10a46518d07'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:43.257531 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-c13b6208-1468-4631-9776-06ef80c67829 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27c5678aa26644629197b10a46518d07'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:43.260172 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-c13b6208-1468-4631-9776-06ef80c67829 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27c5678aa26644629197b10a46518d07'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:43.278494 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 933/1863] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:14:43 2022] DELETE /identity/v3/users/27c5678aa26644629197b10a46518d07 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 12:14:43.633587 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22888b7-5a9e-479a-ac2e-ec6d6bc47d8d admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:43.635799 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-b22888b7-5a9e-479a-ac2e-ec6d6bc47d8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:43.654318 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22888b7-5a9e-479a-ac2e-ec6d6bc47d8d admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:43.787702 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f98b59-2e7b-486d-8be7-fef472652d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:43.790562 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f98b59-2e7b-486d-8be7-fef472652d93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:43.791088 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f98b59-2e7b-486d-8be7-fef472652d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:43.791626 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f98b59-2e7b-486d-8be7-fef472652d93 admin admin] PATH_INFO: `/v3/projects/0ecedd70b8d043668a6b7b01cbd6ee85` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:43.793229 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f98b59-2e7b-486d-8be7-fef472652d93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ecedd70b8d043668a6b7b01cbd6ee85)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:43.795602 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f98b59-2e7b-486d-8be7-fef472652d93 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:43.863322 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-b0f98b59-2e7b-486d-8be7-fef472652d93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ecedd70b8d043668a6b7b01cbd6ee85', 'request_id': 'req-b0f98b59-2e7b-486d-8be7-fef472652d93'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:43.871910 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-b0f98b59-2e7b-486d-8be7-fef472652d93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ecedd70b8d043668a6b7b01cbd6ee85', 'request_id': 'req-b0f98b59-2e7b-486d-8be7-fef472652d93'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:43.880799 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-b0f98b59-2e7b-486d-8be7-fef472652d93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ecedd70b8d043668a6b7b01cbd6ee85', 'request_id': 'req-b0f98b59-2e7b-486d-8be7-fef472652d93'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:43.898354 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-b0f98b59-2e7b-486d-8be7-fef472652d93 admin admin] The token cache is being invalidate because project 0ecedd70b8d043668a6b7b01cbd6ee85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:14:43.898653 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-b0f98b59-2e7b-486d-8be7-fef472652d93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:43.899345 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-b0f98b59-2e7b-486d-8be7-fef472652d93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:43.903246 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 931/1864] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:14:43 2022] DELETE /identity/v3/projects/0ecedd70b8d043668a6b7b01cbd6ee85 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:43.905813 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2dcf9-85c7-4d1a-ba63-c50eb128ca7e None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:43.906104 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2dcf9-85c7-4d1a-ba63-c50eb128ca7e None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:43.906386 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2dcf9-85c7-4d1a-ba63-c50eb128ca7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:43.928851 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-05e2dcf9-85c7-4d1a-ba63-c50eb128ca7e None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:43.930884 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-05e2dcf9-85c7-4d1a-ba63-c50eb128ca7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:43.991664 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 934/1865] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:44.000320 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b77fc7b-c14e-4589-a951-37437846f1ba None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:44.000800 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b77fc7b-c14e-4589-a951-37437846f1ba None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:44.000907 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b77fc7b-c14e-4589-a951-37437846f1ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:44.009540 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:44.022980 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-6b77fc7b-c14e-4589-a951-37437846f1ba None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:44.025136 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-6b77fc7b-c14e-4589-a951-37437846f1ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:44.035842 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 932/1866] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:44.042631 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e2dcf9-85c7-4d1a-ba63-c50eb128ca7e None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:44.054130 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e2dcf9-85c7-4d1a-ba63-c50eb128ca7e None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:44.059947 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79339785-69a5-4ebd-9822-b3a77b5921a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:44.060559 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79339785-69a5-4ebd-9822-b3a77b5921a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:44.060840 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79339785-69a5-4ebd-9822-b3a77b5921a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:44.061114 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79339785-69a5-4ebd-9822-b3a77b5921a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:44.061765 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79339785-69a5-4ebd-9822-b3a77b5921a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:14:44.062067 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79339785-69a5-4ebd-9822-b3a77b5921a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:44.063713 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79339785-69a5-4ebd-9822-b3a77b5921a8 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:44.080098 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 935/1867] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:14:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:14:44.082214 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b77fc7b-c14e-4589-a951-37437846f1ba None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:44.093019 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b77fc7b-c14e-4589-a951-37437846f1ba None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:44.099986 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dcc0ab-1631-4dde-8ff8-6991effce9e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:44.100619 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dcc0ab-1631-4dde-8ff8-6991effce9e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:44.100901 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dcc0ab-1631-4dde-8ff8-6991effce9e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:44.101176 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dcc0ab-1631-4dde-8ff8-6991effce9e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:44.102508 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dcc0ab-1631-4dde-8ff8-6991effce9e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:44.105699 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dcc0ab-1631-4dde-8ff8-6991effce9e0 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:44.124932 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79339785-69a5-4ebd-9822-b3a77b5921a8 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:44.127503 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-79339785-69a5-4ebd-9822-b3a77b5921a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:44.141513 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 933/1868] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:14:44 2022] POST /identity/v3/projects => generated 369 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:14:44.145868 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dcc0ab-1631-4dde-8ff8-6991effce9e0 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:44.150390 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79339785-69a5-4ebd-9822-b3a77b5921a8 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:44.222742 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dcc0ab-1631-4dde-8ff8-6991effce9e0 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:44.230446 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a26d59-333b-4fdf-98a6-ebd55621bcb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:44.231195 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a26d59-333b-4fdf-98a6-ebd55621bcb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:44.231523 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a26d59-333b-4fdf-98a6-ebd55621bcb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:44.231800 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a26d59-333b-4fdf-98a6-ebd55621bcb9 admin admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:44.232951 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfea615b-8a23-4c7c-91dd-f32aa7093740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:44.233093 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a26d59-333b-4fdf-98a6-ebd55621bcb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:44.233745 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea615b-8a23-4c7c-91dd-f32aa7093740 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:44.234096 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea615b-8a23-4c7c-91dd-f32aa7093740 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:44.234442 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea615b-8a23-4c7c-91dd-f32aa7093740 admin admin] PATH_INFO: `/v3/users/b17b06a36ad645808128093781c7b409` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:44.235026 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a26d59-333b-4fdf-98a6-ebd55621bcb9 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:44.238014 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfea615b-8a23-4c7c-91dd-f32aa7093740 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b17b06a36ad645808128093781c7b409)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:44.240528 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfea615b-8a23-4c7c-91dd-f32aa7093740 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:44.276716 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 934/1869] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:14:44 2022] POST /identity/v3/users => generated 285 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:14:44.280465 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a26d59-333b-4fdf-98a6-ebd55621bcb9 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:44.301363 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a26d59-333b-4fdf-98a6-ebd55621bcb9 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:44.313379 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:44.317792 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5d4f38-f079-4c5a-a91b-94b292b5db16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:44.319623 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5d4f38-f079-4c5a-a91b-94b292b5db16 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:44.320022 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5d4f38-f079-4c5a-a91b-94b292b5db16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:44.321661 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5d4f38-f079-4c5a-a91b-94b292b5db16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:44.324496 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5d4f38-f079-4c5a-a91b-94b292b5db16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:44.327780 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5d4f38-f079-4c5a-a91b-94b292b5db16 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:44.333600 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-cfea615b-8a23-4c7c-91dd-f32aa7093740 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b17b06a36ad645808128093781c7b409'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:44.347687 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 935/1870] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:14:44 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:14:44.351169 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-cfea615b-8a23-4c7c-91dd-f32aa7093740 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b17b06a36ad645808128093781c7b409'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:44.352536 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-cfea615b-8a23-4c7c-91dd-f32aa7093740 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b17b06a36ad645808128093781c7b409'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:44.352959 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5d4f38-f079-4c5a-a91b-94b292b5db16 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:44.354651 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-3e5d4f38-f079-4c5a-a91b-94b292b5db16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:44.361443 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-cfea615b-8a23-4c7c-91dd-f32aa7093740 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b17b06a36ad645808128093781c7b409'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:44.362168 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-cfea615b-8a23-4c7c-91dd-f32aa7093740 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b17b06a36ad645808128093781c7b409'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:44.372297 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5d4f38-f079-4c5a-a91b-94b292b5db16 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:44.378139 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcf3345-3fa9-45c0-98ee-aee719e92608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:44.378919 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf3345-3fa9-45c0-98ee-aee719e92608 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:44.379350 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf3345-3fa9-45c0-98ee-aee719e92608 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:44.379730 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf3345-3fa9-45c0-98ee-aee719e92608 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:44.380730 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcf3345-3fa9-45c0-98ee-aee719e92608 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:44.382422 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcf3345-3fa9-45c0-98ee-aee719e92608 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:44.386714 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 936/1871] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:14:44 2022] DELETE /identity/v3/users/b17b06a36ad645808128093781c7b409 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 12:14:44.391308 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906bd064-e99f-4074-a8ed-7a2979720de4 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:44.391665 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906bd064-e99f-4074-a8ed-7a2979720de4 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:44.391961 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906bd064-e99f-4074-a8ed-7a2979720de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:44.397428 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 936/1872] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:14:44 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:14:44.402432 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcf3345-3fa9-45c0-98ee-aee719e92608 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:44.421301 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcf3345-3fa9-45c0-98ee-aee719e92608 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:44.426326 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-906bd064-e99f-4074-a8ed-7a2979720de4 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:44.428835 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-906bd064-e99f-4074-a8ed-7a2979720de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:44.510564 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e817f676-8ab2-460a-b7e1-04b9300dcda2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:44.511281 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e817f676-8ab2-460a-b7e1-04b9300dcda2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:44.511566 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e817f676-8ab2-460a-b7e1-04b9300dcda2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:44.511842 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e817f676-8ab2-460a-b7e1-04b9300dcda2 admin admin] PATH_INFO: `/v3/projects/9fb2ac49cc7846a89254e63fb312ca65/users/0ddb866dc381406a89ae26aee9a94dcd/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:44.533832 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e817f676-8ab2-460a-b7e1-04b9300dcda2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fb2ac49cc7846a89254e63fb312ca65, user_id=0ddb866dc381406a89ae26aee9a94dcd, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:44.533965 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 937/1873] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:44.540760 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e817f676-8ab2-460a-b7e1-04b9300dcda2 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:44.546077 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906bd064-e99f-4074-a8ed-7a2979720de4 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:44.547515 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-906bd064-e99f-4074-a8ed-7a2979720de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:44.579298 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 937/1874] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:14:44 2022] PUT /identity/v3/projects/9fb2ac49cc7846a89254e63fb312ca65/users/0ddb866dc381406a89ae26aee9a94dcd/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:44.581697 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3051c37a-9bdb-47f6-a5cf-4017b01c94b9 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:44.582277 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3051c37a-9bdb-47f6-a5cf-4017b01c94b9 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:44.582810 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3051c37a-9bdb-47f6-a5cf-4017b01c94b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:44.616871 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-3051c37a-9bdb-47f6-a5cf-4017b01c94b9 None None] MFA Rules not processed for user `0ddb866dc381406a89ae26aee9a94dcd`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:44.618672 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906bd064-e99f-4074-a8ed-7a2979720de4 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:44.618818 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-3051c37a-9bdb-47f6-a5cf-4017b01c94b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:44.632376 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:44.682374 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68694ea-8f47-430e-b0ae-01544906bf3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:44.682993 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68694ea-8f47-430e-b0ae-01544906bf3a service neutron] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:44.683301 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68694ea-8f47-430e-b0ae-01544906bf3a service neutron] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:44.683573 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68694ea-8f47-430e-b0ae-01544906bf3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:44.696363 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 938/1875] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:44 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:44.697178 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3051c37a-9bdb-47f6-a5cf-4017b01c94b9 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:44.717225 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:44.754258 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68694ea-8f47-430e-b0ae-01544906bf3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:44.755835 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68694ea-8f47-430e-b0ae-01544906bf3a service neutron] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:44.766855 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 938/1876] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:14:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 12:14:44.786123 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3051c37a-9bdb-47f6-a5cf-4017b01c94b9 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:44.791379 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d60c383-92c3-4f3c-8ee8-667428e47dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:44.792184 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d60c383-92c3-4f3c-8ee8-667428e47dc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:44.793210 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d60c383-92c3-4f3c-8ee8-667428e47dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:44.793653 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d60c383-92c3-4f3c-8ee8-667428e47dc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:44.794439 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d60c383-92c3-4f3c-8ee8-667428e47dc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:44.797863 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d60c383-92c3-4f3c-8ee8-667428e47dc0 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:44.837971 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 939/1877] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:14:44 2022] POST /identity/v3/projects => generated 369 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:14:44.840147 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68694ea-8f47-430e-b0ae-01544906bf3a service neutron] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:44.854470 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68694ea-8f47-430e-b0ae-01544906bf3a service neutron] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:44.961583 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a579bcdf-70e1-47a5-8dbc-301114084781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:44.962295 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579bcdf-70e1-47a5-8dbc-301114084781 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:44.962575 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579bcdf-70e1-47a5-8dbc-301114084781 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:44.962846 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579bcdf-70e1-47a5-8dbc-301114084781 admin admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:44.971134 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a579bcdf-70e1-47a5-8dbc-301114084781 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:44.974181 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a579bcdf-70e1-47a5-8dbc-301114084781 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:45.005567 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 939/1878] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:14:44 2022] POST /identity/v3/users => generated 291 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:14:45.007475 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d60c383-92c3-4f3c-8ee8-667428e47dc0 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:45.019928 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d60c383-92c3-4f3c-8ee8-667428e47dc0 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:45.025705 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cc854a-61e4-42af-a5d6-fab8ce583f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:45.026340 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc854a-61e4-42af-a5d6-fab8ce583f4d admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:45.026624 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc854a-61e4-42af-a5d6-fab8ce583f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:45.026895 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc854a-61e4-42af-a5d6-fab8ce583f4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:45.027606 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cc854a-61e4-42af-a5d6-fab8ce583f4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:45.029222 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cc854a-61e4-42af-a5d6-fab8ce583f4d admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:45.042118 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 940/1879] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:14:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:14:45.057690 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a579bcdf-70e1-47a5-8dbc-301114084781 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:45.071746 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a579bcdf-70e1-47a5-8dbc-301114084781 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:45.094797 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cc854a-61e4-42af-a5d6-fab8ce583f4d admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:45.096672 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-86cc854a-61e4-42af-a5d6-fab8ce583f4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:45.109711 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72fa4a0-2ae2-453a-b8e9-2d4d27a5449c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:45.115280 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72fa4a0-2ae2-453a-b8e9-2d4d27a5449c admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:45.115280 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72fa4a0-2ae2-453a-b8e9-2d4d27a5449c admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:45.115280 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72fa4a0-2ae2-453a-b8e9-2d4d27a5449c admin admin] PATH_INFO: `/v3/projects/ee8ab1e101dd4e13877234ed3dac0cba/users/3100f91e6efb4f14adda83e65c167725/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:45.131344 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cc854a-61e4-42af-a5d6-fab8ce583f4d admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:45.137105 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72fa4a0-2ae2-453a-b8e9-2d4d27a5449c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee8ab1e101dd4e13877234ed3dac0cba, user_id=3100f91e6efb4f14adda83e65c167725, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:45.140367 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72fa4a0-2ae2-453a-b8e9-2d4d27a5449c admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:45.145738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc4f65d-367a-4fc1-909d-beb4b321512f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:45.146790 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc4f65d-367a-4fc1-909d-beb4b321512f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:45.147366 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc4f65d-367a-4fc1-909d-beb4b321512f admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:45.148005 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc4f65d-367a-4fc1-909d-beb4b321512f admin admin] PATH_INFO: `/v3/projects/5623f772dd7540a0971b12e54f9bdf49` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:45.149414 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc4f65d-367a-4fc1-909d-beb4b321512f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5623f772dd7540a0971b12e54f9bdf49)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:45.152007 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc4f65d-367a-4fc1-909d-beb4b321512f admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:45.164423 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 940/1880] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:14:45 2022] PUT /identity/v3/projects/ee8ab1e101dd4e13877234ed3dac0cba/users/3100f91e6efb4f14adda83e65c167725/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:45.168272 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72fa4a0-2ae2-453a-b8e9-2d4d27a5449c admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:45.218550 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-acc4f65d-367a-4fc1-909d-beb4b321512f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5623f772dd7540a0971b12e54f9bdf49', 'request_id': 'req-acc4f65d-367a-4fc1-909d-beb4b321512f'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:45.225801 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-acc4f65d-367a-4fc1-909d-beb4b321512f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5623f772dd7540a0971b12e54f9bdf49', 'request_id': 'req-acc4f65d-367a-4fc1-909d-beb4b321512f'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:45.236721 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-acc4f65d-367a-4fc1-909d-beb4b321512f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5623f772dd7540a0971b12e54f9bdf49', 'request_id': 'req-acc4f65d-367a-4fc1-909d-beb4b321512f'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:45.249021 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72fa4a0-2ae2-453a-b8e9-2d4d27a5449c admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:45.249537 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-acc4f65d-367a-4fc1-909d-beb4b321512f admin admin] The token cache is being invalidate because project 5623f772dd7540a0971b12e54f9bdf49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:14:45.249833 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-acc4f65d-367a-4fc1-909d-beb4b321512f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:45.250552 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-acc4f65d-367a-4fc1-909d-beb4b321512f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:45.255067 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 941/1881] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:14:45 2022] DELETE /identity/v3/projects/5623f772dd7540a0971b12e54f9bdf49 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 12:14:45.260173 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc4f65d-367a-4fc1-909d-beb4b321512f admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:45.263322 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-acc4f65d-367a-4fc1-909d-beb4b321512f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:45.284501 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:45.296627 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:45.329221 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954b8b28-8f35-47eb-a5c4-3c555c3f03ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:45.330173 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b8b28-8f35-47eb-a5c4-3c555c3f03ec admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:45.330523 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b8b28-8f35-47eb-a5c4-3c555c3f03ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:45.330852 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b8b28-8f35-47eb-a5c4-3c555c3f03ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:45.331889 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954b8b28-8f35-47eb-a5c4-3c555c3f03ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:45.334234 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954b8b28-8f35-47eb-a5c4-3c555c3f03ec admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:45.345850 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 941/1882] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:14:45 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 12:14:45.347886 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc4f65d-367a-4fc1-909d-beb4b321512f admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:45.349805 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954b8b28-8f35-47eb-a5c4-3c555c3f03ec admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:45.352087 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-954b8b28-8f35-47eb-a5c4-3c555c3f03ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:45.354295 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c322ab-8515-4270-9955-6750d2b05863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:45.355118 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c322ab-8515-4270-9955-6750d2b05863 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:45.355569 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c322ab-8515-4270-9955-6750d2b05863 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:45.355997 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c322ab-8515-4270-9955-6750d2b05863 admin admin] PATH_INFO: `/v3/users/31003735cf8f4f7098561d3032e97363` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:45.357298 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c322ab-8515-4270-9955-6750d2b05863 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31003735cf8f4f7098561d3032e97363)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:45.359682 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c322ab-8515-4270-9955-6750d2b05863 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:45.377335 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954b8b28-8f35-47eb-a5c4-3c555c3f03ec admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:45.384189 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba472806-9ef9-4d5f-a4a5-1401155d70aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:45.385080 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba472806-9ef9-4d5f-a4a5-1401155d70aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:45.385533 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba472806-9ef9-4d5f-a4a5-1401155d70aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:45.386057 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba472806-9ef9-4d5f-a4a5-1401155d70aa admin admin] PATH_INFO: `/v3/projects/ee8ab1e101dd4e13877234ed3dac0cba/users/3100f91e6efb4f14adda83e65c167725/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:45.388229 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba472806-9ef9-4d5f-a4a5-1401155d70aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee8ab1e101dd4e13877234ed3dac0cba, user_id=3100f91e6efb4f14adda83e65c167725, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:45.390955 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba472806-9ef9-4d5f-a4a5-1401155d70aa admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:45.403578 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 942/1883] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:14:45 2022] PUT /identity/v3/projects/ee8ab1e101dd4e13877234ed3dac0cba/users/3100f91e6efb4f14adda83e65c167725/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 12:14:45.414343 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e8f09-cf2f-4084-a1ef-b7e45e7b59d3 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:45.414640 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e8f09-cf2f-4084-a1ef-b7e45e7b59d3 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:45.415075 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e8f09-cf2f-4084-a1ef-b7e45e7b59d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:45.427377 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-60c322ab-8515-4270-9955-6750d2b05863 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31003735cf8f4f7098561d3032e97363'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:45.436398 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-60c322ab-8515-4270-9955-6750d2b05863 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31003735cf8f4f7098561d3032e97363'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:45.445608 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-60c322ab-8515-4270-9955-6750d2b05863 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31003735cf8f4f7098561d3032e97363'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:45.446142 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-60c322ab-8515-4270-9955-6750d2b05863 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31003735cf8f4f7098561d3032e97363'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:45.446831 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-1f8e8f09-cf2f-4084-a1ef-b7e45e7b59d3 None None] MFA Rules not processed for user `3100f91e6efb4f14adda83e65c167725`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:45.448109 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-1f8e8f09-cf2f-4084-a1ef-b7e45e7b59d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:45.463347 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-60c322ab-8515-4270-9955-6750d2b05863 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31003735cf8f4f7098561d3032e97363'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:45.470386 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 942/1884] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:14:45 2022] DELETE /identity/v3/users/31003735cf8f4f7098561d3032e97363 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:45.538223 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 943/1885] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:45 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:14:45.546353 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c322ab-8515-4270-9955-6750d2b05863 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:45.547918 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-60c322ab-8515-4270-9955-6750d2b05863 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:45.615630 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c322ab-8515-4270-9955-6750d2b05863 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:45.620112 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8541357f-e45f-4dd6-983b-030d8ceee246 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:45.621069 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8541357f-e45f-4dd6-983b-030d8ceee246 service cinder] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:45.621636 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8541357f-e45f-4dd6-983b-030d8ceee246 service cinder] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:45.622098 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8541357f-e45f-4dd6-983b-030d8ceee246 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:45.684544 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:45.707712 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8541357f-e45f-4dd6-983b-030d8ceee246 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:45.709178 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8541357f-e45f-4dd6-983b-030d8ceee246 service cinder] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:45.721678 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 943/1886] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:14:45 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:14:45.779019 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8e8f09-cf2f-4084-a1ef-b7e45e7b59d3 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:45.782611 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8541357f-e45f-4dd6-983b-030d8ceee246 service cinder] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:45.784383 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-8541357f-e45f-4dd6-983b-030d8ceee246 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:45.865497 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8541357f-e45f-4dd6-983b-030d8ceee246 service cinder] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:45.869148 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8e8f09-cf2f-4084-a1ef-b7e45e7b59d3 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:45.872702 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5e87c4-b223-4315-a7a0-7d2ce66772fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:45.872848 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31aef6e-2f3b-4450-85dc-29dc5a901e48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:45.873352 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31aef6e-2f3b-4450-85dc-29dc5a901e48 service cinder] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:45.873632 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31aef6e-2f3b-4450-85dc-29dc5a901e48 service cinder] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:45.873903 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31aef6e-2f3b-4450-85dc-29dc5a901e48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:45.874204 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5e87c4-b223-4315-a7a0-7d2ce66772fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:45.874490 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5e87c4-b223-4315-a7a0-7d2ce66772fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:45.874986 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5e87c4-b223-4315-a7a0-7d2ce66772fa admin admin] PATH_INFO: `/v3/projects/39eabc8708ea47119fa8fbb3f3306966` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:45.876048 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-a31aef6e-2f3b-4450-85dc-29dc5a901e48 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:45.876505 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5e87c4-b223-4315-a7a0-7d2ce66772fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=39eabc8708ea47119fa8fbb3f3306966)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:45.878704 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5e87c4-b223-4315-a7a0-7d2ce66772fa admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:45.941096 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-bd5e87c4-b223-4315-a7a0-7d2ce66772fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39eabc8708ea47119fa8fbb3f3306966', 'request_id': 'req-bd5e87c4-b223-4315-a7a0-7d2ce66772fa'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:45.949084 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-bd5e87c4-b223-4315-a7a0-7d2ce66772fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39eabc8708ea47119fa8fbb3f3306966', 'request_id': 'req-bd5e87c4-b223-4315-a7a0-7d2ce66772fa'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:45.950072 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31aef6e-2f3b-4450-85dc-29dc5a901e48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:45.951663 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31aef6e-2f3b-4450-85dc-29dc5a901e48 service cinder] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:45.957055 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:45.959304 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-bd5e87c4-b223-4315-a7a0-7d2ce66772fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39eabc8708ea47119fa8fbb3f3306966', 'request_id': 'req-bd5e87c4-b223-4315-a7a0-7d2ce66772fa'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:45.965506 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 944/1887] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:14:45 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:14:45.974487 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-bd5e87c4-b223-4315-a7a0-7d2ce66772fa admin admin] The token cache is being invalidate because project 39eabc8708ea47119fa8fbb3f3306966 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:14:45.974805 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-bd5e87c4-b223-4315-a7a0-7d2ce66772fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:45.976137 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-bd5e87c4-b223-4315-a7a0-7d2ce66772fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:45.981845 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 944/1888] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:14:45 2022] DELETE /identity/v3/projects/39eabc8708ea47119fa8fbb3f3306966 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:45.983436 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4543c196-10ca-4843-9e52-073518e81bbc None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:45.983731 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4543c196-10ca-4843-9e52-073518e81bbc None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:45.984013 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4543c196-10ca-4843-9e52-073518e81bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:46.006748 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-4543c196-10ca-4843-9e52-073518e81bbc None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:46.008581 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-4543c196-10ca-4843-9e52-073518e81bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:46.081579 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 945/1889] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:46.082959 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be695843-dc12-4570-863d-ba1aae14ed2f None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:46.083427 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be695843-dc12-4570-863d-ba1aae14ed2f None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:46.083546 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be695843-dc12-4570-863d-ba1aae14ed2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:46.104794 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-be695843-dc12-4570-863d-ba1aae14ed2f None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:46.106016 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-be695843-dc12-4570-863d-ba1aae14ed2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:46.116772 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 945/1890] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:14:46.121630 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4543c196-10ca-4843-9e52-073518e81bbc None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:46.135559 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4543c196-10ca-4843-9e52-073518e81bbc None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:46.141514 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235891bd-dbd6-45f6-9329-bb7bbdaff0b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:46.142131 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235891bd-dbd6-45f6-9329-bb7bbdaff0b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:46.142520 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235891bd-dbd6-45f6-9329-bb7bbdaff0b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:46.142674 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235891bd-dbd6-45f6-9329-bb7bbdaff0b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:46.143368 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235891bd-dbd6-45f6-9329-bb7bbdaff0b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:14:46.143745 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235891bd-dbd6-45f6-9329-bb7bbdaff0b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:46.145409 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235891bd-dbd6-45f6-9329-bb7bbdaff0b1 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:46.158907 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 946/1891] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:14:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:14:46.163007 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be695843-dc12-4570-863d-ba1aae14ed2f None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:46.175933 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be695843-dc12-4570-863d-ba1aae14ed2f None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:46.191495 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4ea0ad-5c56-45b6-9936-dd6b6a5295bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:46.192121 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ea0ad-5c56-45b6-9936-dd6b6a5295bc admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:46.192441 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ea0ad-5c56-45b6-9936-dd6b6a5295bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:46.192671 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ea0ad-5c56-45b6-9936-dd6b6a5295bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:46.193314 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4ea0ad-5c56-45b6-9936-dd6b6a5295bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:46.195319 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4ea0ad-5c56-45b6-9936-dd6b6a5295bc admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:46.230640 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 946/1892] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:14:46 2022] POST /identity/v3/projects => generated 375 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:14:46.235250 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235891bd-dbd6-45f6-9329-bb7bbdaff0b1 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:46.247567 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235891bd-dbd6-45f6-9329-bb7bbdaff0b1 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:46.357149 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c89d2b-4830-4edc-bc23-6325f46fe1f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:46.359779 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c89d2b-4830-4edc-bc23-6325f46fe1f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:46.359779 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c89d2b-4830-4edc-bc23-6325f46fe1f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:46.359779 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c89d2b-4830-4edc-bc23-6325f46fe1f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:46.359779 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c89d2b-4830-4edc-bc23-6325f46fe1f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:46.360819 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c89d2b-4830-4edc-bc23-6325f46fe1f6 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:46.386707 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 947/1893] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:14:46 2022] POST /identity/v3/users => generated 288 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:14:46.390531 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4ea0ad-5c56-45b6-9936-dd6b6a5295bc admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:46.402938 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4ea0ad-5c56-45b6-9936-dd6b6a5295bc admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:46.409761 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd22834-93d0-44db-926c-86d39b020ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:46.410550 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd22834-93d0-44db-926c-86d39b020ef0 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:46.410842 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd22834-93d0-44db-926c-86d39b020ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:46.411173 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd22834-93d0-44db-926c-86d39b020ef0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:46.411819 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd22834-93d0-44db-926c-86d39b020ef0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:46.413426 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd22834-93d0-44db-926c-86d39b020ef0 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:46.427240 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 947/1894] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:14:46.430041 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c89d2b-4830-4edc-bc23-6325f46fe1f6 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:46.440213 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c89d2b-4830-4edc-bc23-6325f46fe1f6 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:46.445041 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c6cd4c-103d-4c66-99bc-3e07b5cf6e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:46.445650 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c6cd4c-103d-4c66-99bc-3e07b5cf6e45 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:46.446005 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c6cd4c-103d-4c66-99bc-3e07b5cf6e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:46.446208 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c6cd4c-103d-4c66-99bc-3e07b5cf6e45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:46.446852 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c6cd4c-103d-4c66-99bc-3e07b5cf6e45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:46.448483 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c6cd4c-103d-4c66-99bc-3e07b5cf6e45 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:46.459820 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 948/1895] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:14:46.464386 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd22834-93d0-44db-926c-86d39b020ef0 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:46.474305 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd22834-93d0-44db-926c-86d39b020ef0 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:46.479624 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad91b37-b163-43c8-84bb-40fec870114b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:46.480277 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad91b37-b163-43c8-84bb-40fec870114b admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:46.480555 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad91b37-b163-43c8-84bb-40fec870114b admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:46.480824 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad91b37-b163-43c8-84bb-40fec870114b admin admin] PATH_INFO: `/v3/projects/a978fc319e6144888abbc0a1d6632cb2/users/68f7291d033b428abf195b06e858fa3f/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:46.496642 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad91b37-b163-43c8-84bb-40fec870114b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a978fc319e6144888abbc0a1d6632cb2, user_id=68f7291d033b428abf195b06e858fa3f, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:46.499387 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad91b37-b163-43c8-84bb-40fec870114b admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:46.514029 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 948/1896] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:14:46 2022] PUT /identity/v3/projects/a978fc319e6144888abbc0a1d6632cb2/users/68f7291d033b428abf195b06e858fa3f/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:46.524433 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a35e568-2a6a-43fe-8ee1-7c4894e8715d None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:46.524720 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a35e568-2a6a-43fe-8ee1-7c4894e8715d None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:46.524995 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a35e568-2a6a-43fe-8ee1-7c4894e8715d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:46.553837 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-6a35e568-2a6a-43fe-8ee1-7c4894e8715d None None] MFA Rules not processed for user `68f7291d033b428abf195b06e858fa3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:46.555314 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-6a35e568-2a6a-43fe-8ee1-7c4894e8715d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:46.618829 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:46.634499 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 949/1897] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:46.639012 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad91b37-b163-43c8-84bb-40fec870114b admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:46.649558 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:46.703101 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad91b37-b163-43c8-84bb-40fec870114b admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:46.707677 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e345bfe-527a-42fd-892c-fda94d5e2306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:46.708267 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e345bfe-527a-42fd-892c-fda94d5e2306 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:46.708544 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e345bfe-527a-42fd-892c-fda94d5e2306 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:46.708811 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e345bfe-527a-42fd-892c-fda94d5e2306 admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:46.709428 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e345bfe-527a-42fd-892c-fda94d5e2306 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:46.711403 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e345bfe-527a-42fd-892c-fda94d5e2306 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:46.737426 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 949/1898] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:14:46 2022] POST /identity/v3/projects => generated 375 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:14:46.746670 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a35e568-2a6a-43fe-8ee1-7c4894e8715d None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:46.760557 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a35e568-2a6a-43fe-8ee1-7c4894e8715d None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:46.831892 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62547397-600b-4113-bf4d-e250af573130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:46.832498 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62547397-600b-4113-bf4d-e250af573130 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:46.832896 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62547397-600b-4113-bf4d-e250af573130 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:46.833172 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62547397-600b-4113-bf4d-e250af573130 admin admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:46.833802 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62547397-600b-4113-bf4d-e250af573130 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:46.836039 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62547397-600b-4113-bf4d-e250af573130 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:46.863846 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 950/1899] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:14:46 2022] POST /identity/v3/users => generated 288 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:14:46.865024 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e345bfe-527a-42fd-892c-fda94d5e2306 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:46.876487 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e345bfe-527a-42fd-892c-fda94d5e2306 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:46.881772 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c72a390-ea4d-4015-af0e-10b99266acf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:46.882512 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c72a390-ea4d-4015-af0e-10b99266acf4 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:46.882885 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c72a390-ea4d-4015-af0e-10b99266acf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:46.883781 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c72a390-ea4d-4015-af0e-10b99266acf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:46.883983 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c72a390-ea4d-4015-af0e-10b99266acf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:46.885580 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c72a390-ea4d-4015-af0e-10b99266acf4 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:46.900123 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 950/1900] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 12:14:46.901337 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62547397-600b-4113-bf4d-e250af573130 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:46.911646 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62547397-600b-4113-bf4d-e250af573130 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:46.917443 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d405f905-f574-40df-a895-326a66320e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:46.918016 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d405f905-f574-40df-a895-326a66320e40 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:46.918292 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d405f905-f574-40df-a895-326a66320e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:46.918862 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d405f905-f574-40df-a895-326a66320e40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:46.919229 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d405f905-f574-40df-a895-326a66320e40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:46.920818 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d405f905-f574-40df-a895-326a66320e40 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:46.935278 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 951/1901] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 12:14:46.936719 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c72a390-ea4d-4015-af0e-10b99266acf4 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:46.948534 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c72a390-ea4d-4015-af0e-10b99266acf4 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:46.953371 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7c344b-579f-4320-97c4-f60624b432d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:46.954164 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c344b-579f-4320-97c4-f60624b432d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:46.954457 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c344b-579f-4320-97c4-f60624b432d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:46.955241 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c344b-579f-4320-97c4-f60624b432d3 admin admin] PATH_INFO: `/v3/projects/85c908b2bdc442b895a30c21201d6deb/users/c76fe081eb6948ab9cdb636269f6abfd/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:46.970830 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7c344b-579f-4320-97c4-f60624b432d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85c908b2bdc442b895a30c21201d6deb, user_id=c76fe081eb6948ab9cdb636269f6abfd, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:46.973516 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7c344b-579f-4320-97c4-f60624b432d3 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:46.990805 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 951/1902] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:14:46 2022] PUT /identity/v3/projects/85c908b2bdc442b895a30c21201d6deb/users/c76fe081eb6948ab9cdb636269f6abfd/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:47.003818 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652897ee-2186-4e10-99cf-c6877b4febe6 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:47.004111 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652897ee-2186-4e10-99cf-c6877b4febe6 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:47.004400 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652897ee-2186-4e10-99cf-c6877b4febe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:47.036857 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-652897ee-2186-4e10-99cf-c6877b4febe6 None None] MFA Rules not processed for user `c76fe081eb6948ab9cdb636269f6abfd`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:47.038243 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-652897ee-2186-4e10-99cf-c6877b4febe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:47.120210 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 952/1903] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:47.124332 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7c344b-579f-4320-97c4-f60624b432d3 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:47.189226 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7c344b-579f-4320-97c4-f60624b432d3 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:47.194509 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9382c1a-903f-46de-9522-4b918a04eaea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:47.195628 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9382c1a-903f-46de-9522-4b918a04eaea admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:47.195925 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9382c1a-903f-46de-9522-4b918a04eaea admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:47.196222 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9382c1a-903f-46de-9522-4b918a04eaea admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:47.197714 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9382c1a-903f-46de-9522-4b918a04eaea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:47.199672 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9382c1a-903f-46de-9522-4b918a04eaea admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:47.233743 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 952/1904] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:14:47 2022] POST /identity/v3/projects => generated 377 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:14:47.234783 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652897ee-2186-4e10-99cf-c6877b4febe6 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:47.246154 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652897ee-2186-4e10-99cf-c6877b4febe6 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:47.305678 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926a1db3-c705-44d8-8666-5ea57a683f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:47.306294 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926a1db3-c705-44d8-8666-5ea57a683f18 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:47.306580 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926a1db3-c705-44d8-8666-5ea57a683f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:47.306852 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926a1db3-c705-44d8-8666-5ea57a683f18 admin admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:47.308035 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926a1db3-c705-44d8-8666-5ea57a683f18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:47.309429 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926a1db3-c705-44d8-8666-5ea57a683f18 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:47.320887 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:47.348831 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 953/1905] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:14:47 2022] POST /identity/v3/users => generated 295 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 12:14:47.354949 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9382c1a-903f-46de-9522-4b918a04eaea admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:47.365248 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9382c1a-903f-46de-9522-4b918a04eaea admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:47.372045 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ff8ed1-61c4-49a2-8472-dec82544c58e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:47.372603 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff8ed1-61c4-49a2-8472-dec82544c58e admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:47.372884 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff8ed1-61c4-49a2-8472-dec82544c58e admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:47.373157 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff8ed1-61c4-49a2-8472-dec82544c58e admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:47.373806 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ff8ed1-61c4-49a2-8472-dec82544c58e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:47.375609 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ff8ed1-61c4-49a2-8472-dec82544c58e admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:47.391441 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 953/1906] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:14:47.394771 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926a1db3-c705-44d8-8666-5ea57a683f18 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:47.419237 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926a1db3-c705-44d8-8666-5ea57a683f18 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:47.425005 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45bb672-813d-4560-aa18-450d1262b54b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:47.425668 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45bb672-813d-4560-aa18-450d1262b54b admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:47.426138 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45bb672-813d-4560-aa18-450d1262b54b admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:47.426419 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45bb672-813d-4560-aa18-450d1262b54b admin admin] PATH_INFO: `/v3/projects/7f7f322acdca49e28fc25e70044dbcf6/users/1cd7b58d61ed404694f73235a3d8c286/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:47.463381 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45bb672-813d-4560-aa18-450d1262b54b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f7f322acdca49e28fc25e70044dbcf6, user_id=1cd7b58d61ed404694f73235a3d8c286, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:47.466234 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45bb672-813d-4560-aa18-450d1262b54b admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:47.491371 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 954/1907] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:14:47 2022] PUT /identity/v3/projects/7f7f322acdca49e28fc25e70044dbcf6/users/1cd7b58d61ed404694f73235a3d8c286/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 12:14:47.493385 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ff8ed1-61c4-49a2-8472-dec82544c58e admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:47.510140 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ff8ed1-61c4-49a2-8472-dec82544c58e admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:47.561440 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:47.616382 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83d4c58-be8b-45d0-b072-9edfb741d109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:47.617626 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d4c58-be8b-45d0-b072-9edfb741d109 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:47.617910 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d4c58-be8b-45d0-b072-9edfb741d109 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:47.618187 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d4c58-be8b-45d0-b072-9edfb741d109 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:47.618835 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83d4c58-be8b-45d0-b072-9edfb741d109 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:47.620476 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83d4c58-be8b-45d0-b072-9edfb741d109 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:47.631232 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 954/1908] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 12:14:47.635591 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45bb672-813d-4560-aa18-450d1262b54b admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:47.648953 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45bb672-813d-4560-aa18-450d1262b54b admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:47.654156 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d4e0cd-0690-41ef-be5b-ea0835e60d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:47.654820 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d4e0cd-0690-41ef-be5b-ea0835e60d57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:47.655100 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d4e0cd-0690-41ef-be5b-ea0835e60d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:47.655404 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d4e0cd-0690-41ef-be5b-ea0835e60d57 admin admin] PATH_INFO: `/v3/projects/7f7f322acdca49e28fc25e70044dbcf6/users/1cd7b58d61ed404694f73235a3d8c286/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:47.657362 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d4e0cd-0690-41ef-be5b-ea0835e60d57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f7f322acdca49e28fc25e70044dbcf6, user_id=1cd7b58d61ed404694f73235a3d8c286, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:47.660295 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d4e0cd-0690-41ef-be5b-ea0835e60d57 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:47.675241 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 955/1909] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:14:47 2022] PUT /identity/v3/projects/7f7f322acdca49e28fc25e70044dbcf6/users/1cd7b58d61ed404694f73235a3d8c286/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 12:14:47.684449 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8031e44-0f49-4693-afec-c1fcb7f8f855 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:47.684743 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8031e44-0f49-4693-afec-c1fcb7f8f855 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:47.685023 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8031e44-0f49-4693-afec-c1fcb7f8f855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:47.734844 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-e8031e44-0f49-4693-afec-c1fcb7f8f855 None None] MFA Rules not processed for user `1cd7b58d61ed404694f73235a3d8c286`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:47.736134 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-e8031e44-0f49-4693-afec-c1fcb7f8f855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:47.816116 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 955/1910] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:47 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:47.830008 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d4e0cd-0690-41ef-be5b-ea0835e60d57 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:47.832114 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-a8d4e0cd-0690-41ef-be5b-ea0835e60d57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:47.940869 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d4e0cd-0690-41ef-be5b-ea0835e60d57 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:47.944836 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bf3d40-0a27-45be-b129-bfda7eb6f118 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:47.945568 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf3d40-0a27-45be-b129-bfda7eb6f118 service cinder] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:47.946052 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf3d40-0a27-45be-b129-bfda7eb6f118 service cinder] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:47.946333 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf3d40-0a27-45be-b129-bfda7eb6f118 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:48.004468 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bf3d40-0a27-45be-b129-bfda7eb6f118 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:48.005905 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bf3d40-0a27-45be-b129-bfda7eb6f118 service cinder] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:48.014874 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 956/1911] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:14:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:14:49.526903 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8031e44-0f49-4693-afec-c1fcb7f8f855 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:49.528396 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-e8031e44-0f49-4693-afec-c1fcb7f8f855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:49.638492 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8031e44-0f49-4693-afec-c1fcb7f8f855 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:49.643520 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b5a891-311c-4555-8bf1-93f71efbd38a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:49.644178 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b5a891-311c-4555-8bf1-93f71efbd38a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:49.644468 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b5a891-311c-4555-8bf1-93f71efbd38a admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:49.644743 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b5a891-311c-4555-8bf1-93f71efbd38a admin admin] PATH_INFO: `/v3/users/0ddb866dc381406a89ae26aee9a94dcd` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:49.646403 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b5a891-311c-4555-8bf1-93f71efbd38a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ddb866dc381406a89ae26aee9a94dcd)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:49.649112 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b5a891-311c-4555-8bf1-93f71efbd38a admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:49.730475 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-73b5a891-311c-4555-8bf1-93f71efbd38a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ddb866dc381406a89ae26aee9a94dcd'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:49.737695 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-73b5a891-311c-4555-8bf1-93f71efbd38a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ddb866dc381406a89ae26aee9a94dcd'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:49.745343 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:49.746820 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-73b5a891-311c-4555-8bf1-93f71efbd38a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ddb866dc381406a89ae26aee9a94dcd'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:49.747522 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-73b5a891-311c-4555-8bf1-93f71efbd38a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ddb866dc381406a89ae26aee9a94dcd'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:49.760823 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-73b5a891-311c-4555-8bf1-93f71efbd38a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ddb866dc381406a89ae26aee9a94dcd'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:49.766604 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 956/1912] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:14:49 2022] DELETE /identity/v3/users/0ddb866dc381406a89ae26aee9a94dcd => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:49.771485 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9fb55b-9099-4259-ad06-effa5ba7bf68 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:49.771935 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9fb55b-9099-4259-ad06-effa5ba7bf68 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:49.772061 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9fb55b-9099-4259-ad06-effa5ba7bf68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:49.790912 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-bd9fb55b-9099-4259-ad06-effa5ba7bf68 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:49.792151 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-bd9fb55b-9099-4259-ad06-effa5ba7bf68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:49.853713 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 957/1913] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:49.863336 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b5a891-311c-4555-8bf1-93f71efbd38a admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:49.865005 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-73b5a891-311c-4555-8bf1-93f71efbd38a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:49.937849 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b5a891-311c-4555-8bf1-93f71efbd38a admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:49.941743 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6553b53d-f473-4fc9-9858-efaeb28db90a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:49.942366 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6553b53d-f473-4fc9-9858-efaeb28db90a service neutron] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:49.942648 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6553b53d-f473-4fc9-9858-efaeb28db90a service neutron] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:49.942921 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6553b53d-f473-4fc9-9858-efaeb28db90a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:49.952975 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6553b53d-f473-4fc9-9858-efaeb28db90a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:49.954411 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6553b53d-f473-4fc9-9858-efaeb28db90a service neutron] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:49.964130 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 957/1914] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:14:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:14:50.413030 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9fb55b-9099-4259-ad06-effa5ba7bf68 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:50.415179 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-bd9fb55b-9099-4259-ad06-effa5ba7bf68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:50.427045 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9fb55b-9099-4259-ad06-effa5ba7bf68 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:50.431703 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577da142-4734-4fc2-a557-f6f0ebdfd103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:50.432329 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577da142-4734-4fc2-a557-f6f0ebdfd103 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:50.432602 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577da142-4734-4fc2-a557-f6f0ebdfd103 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:50.432869 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577da142-4734-4fc2-a557-f6f0ebdfd103 admin admin] PATH_INFO: `/v3/projects/9fb2ac49cc7846a89254e63fb312ca65` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:50.433968 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577da142-4734-4fc2-a557-f6f0ebdfd103 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fb2ac49cc7846a89254e63fb312ca65)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:50.435901 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577da142-4734-4fc2-a557-f6f0ebdfd103 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:50.460294 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:50.504649 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-577da142-4734-4fc2-a557-f6f0ebdfd103 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fb2ac49cc7846a89254e63fb312ca65', 'request_id': 'req-577da142-4734-4fc2-a557-f6f0ebdfd103'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:50.511148 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-577da142-4734-4fc2-a557-f6f0ebdfd103 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fb2ac49cc7846a89254e63fb312ca65', 'request_id': 'req-577da142-4734-4fc2-a557-f6f0ebdfd103'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:50.518844 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-577da142-4734-4fc2-a557-f6f0ebdfd103 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fb2ac49cc7846a89254e63fb312ca65', 'request_id': 'req-577da142-4734-4fc2-a557-f6f0ebdfd103'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:50.529905 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-577da142-4734-4fc2-a557-f6f0ebdfd103 admin admin] The token cache is being invalidate because project 9fb2ac49cc7846a89254e63fb312ca65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81712) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:14:50.530191 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-577da142-4734-4fc2-a557-f6f0ebdfd103 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:50.530819 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-577da142-4734-4fc2-a557-f6f0ebdfd103 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:50.532353 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 958/1915] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:14:50 2022] DELETE /identity/v3/projects/9fb2ac49cc7846a89254e63fb312ca65 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:50.536516 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6553b53d-f473-4fc9-9858-efaeb28db90a service neutron] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:50.546668 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6553b53d-f473-4fc9-9858-efaeb28db90a service neutron] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:50.601470 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa9100a-d66a-4ad3-8584-9ef002d5c854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:50.602022 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa9100a-d66a-4ad3-8584-9ef002d5c854 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:50.602296 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa9100a-d66a-4ad3-8584-9ef002d5c854 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:50.602566 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa9100a-d66a-4ad3-8584-9ef002d5c854 admin admin] PATH_INFO: `/v3/users/3100f91e6efb4f14adda83e65c167725` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:50.603728 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa9100a-d66a-4ad3-8584-9ef002d5c854 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3100f91e6efb4f14adda83e65c167725)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:50.605699 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa9100a-d66a-4ad3-8584-9ef002d5c854 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:50.652799 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-6fa9100a-d66a-4ad3-8584-9ef002d5c854 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3100f91e6efb4f14adda83e65c167725'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:50.660582 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-6fa9100a-d66a-4ad3-8584-9ef002d5c854 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3100f91e6efb4f14adda83e65c167725'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:50.668944 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-6fa9100a-d66a-4ad3-8584-9ef002d5c854 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3100f91e6efb4f14adda83e65c167725'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:50.669636 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-6fa9100a-d66a-4ad3-8584-9ef002d5c854 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3100f91e6efb4f14adda83e65c167725'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:50.682279 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:50.684216 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-6fa9100a-d66a-4ad3-8584-9ef002d5c854 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3100f91e6efb4f14adda83e65c167725'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:50.699292 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 958/1916] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:14:50 2022] DELETE /identity/v3/users/3100f91e6efb4f14adda83e65c167725 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:50.962200 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577da142-4734-4fc2-a557-f6f0ebdfd103 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:50.963650 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-577da142-4734-4fc2-a557-f6f0ebdfd103 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:50.978964 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577da142-4734-4fc2-a557-f6f0ebdfd103 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:51.037514 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6eebb3-8923-4c10-9d73-9244548094b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:51.038145 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6eebb3-8923-4c10-9d73-9244548094b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:51.038427 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6eebb3-8923-4c10-9d73-9244548094b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:51.038744 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6eebb3-8923-4c10-9d73-9244548094b1 admin admin] PATH_INFO: `/v3/projects/ee8ab1e101dd4e13877234ed3dac0cba` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:51.040173 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6eebb3-8923-4c10-9d73-9244548094b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee8ab1e101dd4e13877234ed3dac0cba)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:51.042085 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6eebb3-8923-4c10-9d73-9244548094b1 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:51.088923 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-dc6eebb3-8923-4c10-9d73-9244548094b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee8ab1e101dd4e13877234ed3dac0cba', 'request_id': 'req-dc6eebb3-8923-4c10-9d73-9244548094b1'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:51.096478 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-dc6eebb3-8923-4c10-9d73-9244548094b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee8ab1e101dd4e13877234ed3dac0cba', 'request_id': 'req-dc6eebb3-8923-4c10-9d73-9244548094b1'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:51.105396 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-dc6eebb3-8923-4c10-9d73-9244548094b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee8ab1e101dd4e13877234ed3dac0cba', 'request_id': 'req-dc6eebb3-8923-4c10-9d73-9244548094b1'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:51.118787 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-dc6eebb3-8923-4c10-9d73-9244548094b1 admin admin] The token cache is being invalidate because project ee8ab1e101dd4e13877234ed3dac0cba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81712) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:14:51.119081 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-dc6eebb3-8923-4c10-9d73-9244548094b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:51.119788 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-dc6eebb3-8923-4c10-9d73-9244548094b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:51.121462 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 959/1917] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:14:50 2022] DELETE /identity/v3/projects/ee8ab1e101dd4e13877234ed3dac0cba => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:51.126326 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7b7e45-07c8-4149-9a3c-778d822f309b None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:51.126326 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7b7e45-07c8-4149-9a3c-778d822f309b None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:51.126565 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7b7e45-07c8-4149-9a3c-778d822f309b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:51.147214 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-ae7b7e45-07c8-4149-9a3c-778d822f309b None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:51.148362 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-ae7b7e45-07c8-4149-9a3c-778d822f309b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:51.218391 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 959/1918] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:51.227488 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef682c-afa0-4e52-aaee-fa703ee82a4b None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:51.227772 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef682c-afa0-4e52-aaee-fa703ee82a4b None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:51.228059 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef682c-afa0-4e52-aaee-fa703ee82a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:51.250138 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-50ef682c-afa0-4e52-aaee-fa703ee82a4b None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:51.251402 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-50ef682c-afa0-4e52-aaee-fa703ee82a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:51.260185 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 960/1919] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:51.264199 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7b7e45-07c8-4149-9a3c-778d822f309b None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:51.274231 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7b7e45-07c8-4149-9a3c-778d822f309b None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:51.279042 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb603da-bb61-493f-92a0-68a39724c3ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:51.279674 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb603da-bb61-493f-92a0-68a39724c3ad admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:51.279952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb603da-bb61-493f-92a0-68a39724c3ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:51.280221 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb603da-bb61-493f-92a0-68a39724c3ad admin admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:51.280863 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb603da-bb61-493f-92a0-68a39724c3ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:14:51.281163 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb603da-bb61-493f-92a0-68a39724c3ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:51.282747 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb603da-bb61-493f-92a0-68a39724c3ad admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:51.297839 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 960/1920] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:14:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:14:51.298951 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ef682c-afa0-4e52-aaee-fa703ee82a4b None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:51.308943 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ef682c-afa0-4e52-aaee-fa703ee82a4b None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:51.318943 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff333d24-4824-4d39-8b9d-82cfd9215346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:51.319567 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff333d24-4824-4d39-8b9d-82cfd9215346 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:51.319940 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff333d24-4824-4d39-8b9d-82cfd9215346 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:51.320741 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff333d24-4824-4d39-8b9d-82cfd9215346 admin admin] PATH_INFO: `/v3/projects` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:51.321369 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff333d24-4824-4d39-8b9d-82cfd9215346 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:51.323293 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff333d24-4824-4d39-8b9d-82cfd9215346 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:51.338509 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:51.352773 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 961/1921] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:14:51 2022] POST /identity/v3/projects => generated 379 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 12:14:51.356916 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb603da-bb61-493f-92a0-68a39724c3ad admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:51.366735 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb603da-bb61-493f-92a0-68a39724c3ad admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:51.421317 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6f1126-5cd9-42f2-aa37-a63d288d60e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:51.421620 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f1126-5cd9-42f2-aa37-a63d288d60e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:51.421898 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f1126-5cd9-42f2-aa37-a63d288d60e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:51.422167 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f1126-5cd9-42f2-aa37-a63d288d60e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:51.423160 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6f1126-5cd9-42f2-aa37-a63d288d60e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:51.424742 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6f1126-5cd9-42f2-aa37-a63d288d60e8 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:51.447522 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 961/1922] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:14:51 2022] POST /identity/v3/users => generated 290 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:14:51.451703 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff333d24-4824-4d39-8b9d-82cfd9215346 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:51.463370 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff333d24-4824-4d39-8b9d-82cfd9215346 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:51.468597 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f5ff4-32c1-4566-9846-e5e9a92f5044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:51.469286 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f5ff4-32c1-4566-9846-e5e9a92f5044 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:51.469565 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f5ff4-32c1-4566-9846-e5e9a92f5044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:51.469835 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f5ff4-32c1-4566-9846-e5e9a92f5044 admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:51.470474 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998f5ff4-32c1-4566-9846-e5e9a92f5044 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:51.472222 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998f5ff4-32c1-4566-9846-e5e9a92f5044 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:51.487848 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6f1126-5cd9-42f2-aa37-a63d288d60e8 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:51.493751 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 962/1923] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:14:51 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 12:14:51.501411 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6f1126-5cd9-42f2-aa37-a63d288d60e8 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:51.507869 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb77525a-82ed-4ea3-8bd1-a509b1c3079a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:51.508700 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb77525a-82ed-4ea3-8bd1-a509b1c3079a admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:51.509049 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb77525a-82ed-4ea3-8bd1-a509b1c3079a admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:51.509389 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb77525a-82ed-4ea3-8bd1-a509b1c3079a admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:51.510102 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb77525a-82ed-4ea3-8bd1-a509b1c3079a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:51.511816 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb77525a-82ed-4ea3-8bd1-a509b1c3079a admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:51.523786 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 962/1924] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:14:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 12:14:51.527978 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f5ff4-32c1-4566-9846-e5e9a92f5044 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:51.537781 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998f5ff4-32c1-4566-9846-e5e9a92f5044 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:51.542915 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23207f25-901b-4f0e-8c37-59a39912c3d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:51.543681 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23207f25-901b-4f0e-8c37-59a39912c3d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:51.544023 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23207f25-901b-4f0e-8c37-59a39912c3d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:51.544596 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23207f25-901b-4f0e-8c37-59a39912c3d8 admin admin] PATH_INFO: `/v3/projects/883e9e5cec154ac79fdeab55e2ddf282/users/b525d07d26154a6aa3749328c080881f/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:51.560405 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23207f25-901b-4f0e-8c37-59a39912c3d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=883e9e5cec154ac79fdeab55e2ddf282, user_id=b525d07d26154a6aa3749328c080881f, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:51.563078 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23207f25-901b-4f0e-8c37-59a39912c3d8 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:51.579157 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 963/1925] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:14:51 2022] PUT /identity/v3/projects/883e9e5cec154ac79fdeab55e2ddf282/users/b525d07d26154a6aa3749328c080881f/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 12:14:51.590490 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd6dd02-ec66-41cb-84cb-c5668fa9723c None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:51.590942 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd6dd02-ec66-41cb-84cb-c5668fa9723c None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:51.591027 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd6dd02-ec66-41cb-84cb-c5668fa9723c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:51.619991 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-2fd6dd02-ec66-41cb-84cb-c5668fa9723c None None] MFA Rules not processed for user `b525d07d26154a6aa3749328c080881f`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:51.621193 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-2fd6dd02-ec66-41cb-84cb-c5668fa9723c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:51.676386 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:51.700277 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 963/1926] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:51 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:51.704377 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23207f25-901b-4f0e-8c37-59a39912c3d8 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:51.774266 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23207f25-901b-4f0e-8c37-59a39912c3d8 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:51.779137 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd18c19-47bf-4a37-ac95-fd5c9592dfd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:51.779767 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd18c19-47bf-4a37-ac95-fd5c9592dfd3 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:51.780043 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd18c19-47bf-4a37-ac95-fd5c9592dfd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:51.780310 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd18c19-47bf-4a37-ac95-fd5c9592dfd3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:51.780960 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd18c19-47bf-4a37-ac95-fd5c9592dfd3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:51.782951 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd18c19-47bf-4a37-ac95-fd5c9592dfd3 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:51.809999 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 964/1927] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:14:51 2022] POST /identity/v3/projects => generated 381 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:14:51.814057 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd6dd02-ec66-41cb-84cb-c5668fa9723c None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:51.825403 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd6dd02-ec66-41cb-84cb-c5668fa9723c None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:51.880131 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47859570-fc97-4f35-95f6-b9721b988d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:51.880734 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47859570-fc97-4f35-95f6-b9721b988d2d admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:51.881016 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47859570-fc97-4f35-95f6-b9721b988d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:51.881294 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47859570-fc97-4f35-95f6-b9721b988d2d admin admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:51.881913 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47859570-fc97-4f35-95f6-b9721b988d2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:51.884605 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47859570-fc97-4f35-95f6-b9721b988d2d admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:51.905394 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 964/1928] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:14:51 2022] POST /identity/v3/users => generated 297 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:14:51.909604 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd18c19-47bf-4a37-ac95-fd5c9592dfd3 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:51.919992 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd18c19-47bf-4a37-ac95-fd5c9592dfd3 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:51.924645 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de205b0-2d87-4e94-851c-741b569ae9d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:51.925228 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de205b0-2d87-4e94-851c-741b569ae9d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:51.925504 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de205b0-2d87-4e94-851c-741b569ae9d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:51.925776 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de205b0-2d87-4e94-851c-741b569ae9d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:51.926506 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de205b0-2d87-4e94-851c-741b569ae9d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:51.928114 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de205b0-2d87-4e94-851c-741b569ae9d6 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:51.938943 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 965/1929] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:14:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:14:51.942906 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47859570-fc97-4f35-95f6-b9721b988d2d admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:51.953374 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47859570-fc97-4f35-95f6-b9721b988d2d admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:51.957942 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcf8b75-285b-4720-b9bc-d5ace03d3265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:51.958589 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcf8b75-285b-4720-b9bc-d5ace03d3265 admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:51.958870 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcf8b75-285b-4720-b9bc-d5ace03d3265 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:51.959145 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcf8b75-285b-4720-b9bc-d5ace03d3265 admin admin] PATH_INFO: `/v3/projects/6f83ad913d9543d3a7d06665590c1998/users/2e71e248c58b446186399dcdb0a03ee0/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:51.973818 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcf8b75-285b-4720-b9bc-d5ace03d3265 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f83ad913d9543d3a7d06665590c1998, user_id=2e71e248c58b446186399dcdb0a03ee0, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:51.976651 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcf8b75-285b-4720-b9bc-d5ace03d3265 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:51.989205 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 965/1930] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:14:51 2022] PUT /identity/v3/projects/6f83ad913d9543d3a7d06665590c1998/users/2e71e248c58b446186399dcdb0a03ee0/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:51.993952 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de205b0-2d87-4e94-851c-741b569ae9d6 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:52.004148 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de205b0-2d87-4e94-851c-741b569ae9d6 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:52.060316 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35217ff6-6959-48b5-97a8-1d2bc36abe3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:52.060913 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35217ff6-6959-48b5-97a8-1d2bc36abe3e admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:52.061191 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35217ff6-6959-48b5-97a8-1d2bc36abe3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:52.061462 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35217ff6-6959-48b5-97a8-1d2bc36abe3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:52.062090 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35217ff6-6959-48b5-97a8-1d2bc36abe3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:52.063701 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35217ff6-6959-48b5-97a8-1d2bc36abe3e admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:52.074035 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 966/1931] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:14:51 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 12:14:52.078152 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcf8b75-285b-4720-b9bc-d5ace03d3265 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:52.088409 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcf8b75-285b-4720-b9bc-d5ace03d3265 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:52.092875 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e1499b-4352-4c90-bc27-17750c96a4e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:52.093508 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1499b-4352-4c90-bc27-17750c96a4e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:52.093779 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1499b-4352-4c90-bc27-17750c96a4e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:52.094057 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1499b-4352-4c90-bc27-17750c96a4e9 admin admin] PATH_INFO: `/v3/projects/6f83ad913d9543d3a7d06665590c1998/users/2e71e248c58b446186399dcdb0a03ee0/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:52.095981 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e1499b-4352-4c90-bc27-17750c96a4e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f83ad913d9543d3a7d06665590c1998, user_id=2e71e248c58b446186399dcdb0a03ee0, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:52.098639 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e1499b-4352-4c90-bc27-17750c96a4e9 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:52.109032 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 966/1932] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:14:52 2022] PUT /identity/v3/projects/6f83ad913d9543d3a7d06665590c1998/users/2e71e248c58b446186399dcdb0a03ee0/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:52.119810 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f3e5b-f9ec-4601-bb10-c41a95612ef0 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:52.120096 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f3e5b-f9ec-4601-bb10-c41a95612ef0 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:52.120365 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f3e5b-f9ec-4601-bb10-c41a95612ef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:52.147229 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-a10f3e5b-f9ec-4601-bb10-c41a95612ef0 None None] MFA Rules not processed for user `2e71e248c58b446186399dcdb0a03ee0`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:52.148617 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-a10f3e5b-f9ec-4601-bb10-c41a95612ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:52.166760 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:52.221521 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 967/1933] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:52 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:52.234388 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e1499b-4352-4c90-bc27-17750c96a4e9 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:52.236106 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-16e1499b-4352-4c90-bc27-17750c96a4e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:52.292767 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e1499b-4352-4c90-bc27-17750c96a4e9 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:52.296370 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1ed895-22df-428a-9a12-c6eb7ec33153 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:52.296958 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ed895-22df-428a-9a12-c6eb7ec33153 service cinder] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:52.297229 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ed895-22df-428a-9a12-c6eb7ec33153 service cinder] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:52.297495 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1ed895-22df-428a-9a12-c6eb7ec33153 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:52.306929 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1ed895-22df-428a-9a12-c6eb7ec33153 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:52.308402 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1ed895-22df-428a-9a12-c6eb7ec33153 service cinder] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:52.318296 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 967/1934] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:14:52 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:14:52.665292 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10f3e5b-f9ec-4601-bb10-c41a95612ef0 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:52.674328 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10f3e5b-f9ec-4601-bb10-c41a95612ef0 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:52.678957 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3986f23e-c836-4bb3-9f40-a7331d6f637d tempest-VolumeQuotaClassesTest-1774721437 tempest-VolumeQuotaClassesTest-1774721437-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71e248c58b446186399dcdb0a03ee0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f83ad913d9543d3a7d06665590c1998', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:52.680133 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986f23e-c836-4bb3-9f40-a7331d6f637d tempest-VolumeQuotaClassesTest-1774721437 tempest-VolumeQuotaClassesTest-1774721437-project-admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:52.680417 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986f23e-c836-4bb3-9f40-a7331d6f637d tempest-VolumeQuotaClassesTest-1774721437 tempest-VolumeQuotaClassesTest-1774721437-project-admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:52.680693 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986f23e-c836-4bb3-9f40-a7331d6f637d tempest-VolumeQuotaClassesTest-1774721437 tempest-VolumeQuotaClassesTest-1774721437-project-admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:52.681343 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3986f23e-c836-4bb3-9f40-a7331d6f637d tempest-VolumeQuotaClassesTest-1774721437 tempest-VolumeQuotaClassesTest-1774721437-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:14:52.681646 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3986f23e-c836-4bb3-9f40-a7331d6f637d tempest-VolumeQuotaClassesTest-1774721437 tempest-VolumeQuotaClassesTest-1774721437-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:52.683253 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3986f23e-c836-4bb3-9f40-a7331d6f637d tempest-VolumeQuotaClassesTest-1774721437 tempest-VolumeQuotaClassesTest-1774721437-project-admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:52.694576 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 968/1935] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:14:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:14:52.699560 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1ed895-22df-428a-9a12-c6eb7ec33153 service cinder] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:52.708982 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1ed895-22df-428a-9a12-c6eb7ec33153 service cinder] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:52.713951 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e0edf-6c32-4e99-a076-7691831b153b tempest-VolumeQuotaClassesTest-1774721437 tempest-VolumeQuotaClassesTest-1774721437-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71e248c58b446186399dcdb0a03ee0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f83ad913d9543d3a7d06665590c1998', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:52.714613 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e0edf-6c32-4e99-a076-7691831b153b tempest-VolumeQuotaClassesTest-1774721437 tempest-VolumeQuotaClassesTest-1774721437-project-admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:52.714893 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e0edf-6c32-4e99-a076-7691831b153b tempest-VolumeQuotaClassesTest-1774721437 tempest-VolumeQuotaClassesTest-1774721437-project-admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:52.715164 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e0edf-6c32-4e99-a076-7691831b153b tempest-VolumeQuotaClassesTest-1774721437 tempest-VolumeQuotaClassesTest-1774721437-project-admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:52.717274 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250e0edf-6c32-4e99-a076-7691831b153b tempest-VolumeQuotaClassesTest-1774721437 tempest-VolumeQuotaClassesTest-1774721437-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:52.719163 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250e0edf-6c32-4e99-a076-7691831b153b tempest-VolumeQuotaClassesTest-1774721437 tempest-VolumeQuotaClassesTest-1774721437-project-admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:52.762820 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 968/1936] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:14:52 2022] POST /identity/v3/projects => generated 354 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:14:52.766581 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3986f23e-c836-4bb3-9f40-a7331d6f637d tempest-VolumeQuotaClassesTest-1774721437 tempest-VolumeQuotaClassesTest-1774721437-project-admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:52.779162 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3986f23e-c836-4bb3-9f40-a7331d6f637d tempest-VolumeQuotaClassesTest-1774721437 tempest-VolumeQuotaClassesTest-1774721437-project-admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:52.845902 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fee1c9-7c2d-400c-8e96-ad6eaaa4f537 tempest-VolumeQuotaClassesTest-1774721437 tempest-VolumeQuotaClassesTest-1774721437-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71e248c58b446186399dcdb0a03ee0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f83ad913d9543d3a7d06665590c1998', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:52.846625 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fee1c9-7c2d-400c-8e96-ad6eaaa4f537 tempest-VolumeQuotaClassesTest-1774721437 tempest-VolumeQuotaClassesTest-1774721437-project-admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:52.847025 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fee1c9-7c2d-400c-8e96-ad6eaaa4f537 tempest-VolumeQuotaClassesTest-1774721437 tempest-VolumeQuotaClassesTest-1774721437-project-admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:52.847433 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fee1c9-7c2d-400c-8e96-ad6eaaa4f537 tempest-VolumeQuotaClassesTest-1774721437 tempest-VolumeQuotaClassesTest-1774721437-project-admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:52.848186 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fee1c9-7c2d-400c-8e96-ad6eaaa4f537 tempest-VolumeQuotaClassesTest-1774721437 tempest-VolumeQuotaClassesTest-1774721437-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:14:52.848595 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fee1c9-7c2d-400c-8e96-ad6eaaa4f537 tempest-VolumeQuotaClassesTest-1774721437 tempest-VolumeQuotaClassesTest-1774721437-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:52.850394 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fee1c9-7c2d-400c-8e96-ad6eaaa4f537 tempest-VolumeQuotaClassesTest-1774721437 tempest-VolumeQuotaClassesTest-1774721437-project-admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:52.863855 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 969/1937] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:14:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:14:52.904666 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e0edf-6c32-4e99-a076-7691831b153b tempest-VolumeQuotaClassesTest-1774721437 tempest-VolumeQuotaClassesTest-1774721437-project-admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:52.915555 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e0edf-6c32-4e99-a076-7691831b153b tempest-VolumeQuotaClassesTest-1774721437 tempest-VolumeQuotaClassesTest-1774721437-project-admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:52.920190 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627bb68f-9ac7-4f55-ad3d-b57db6a68378 tempest-VolumeQuotaClassesTest-1774721437 tempest-VolumeQuotaClassesTest-1774721437-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e71e248c58b446186399dcdb0a03ee0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f83ad913d9543d3a7d06665590c1998', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:52.920822 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627bb68f-9ac7-4f55-ad3d-b57db6a68378 tempest-VolumeQuotaClassesTest-1774721437 tempest-VolumeQuotaClassesTest-1774721437-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:52.921107 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627bb68f-9ac7-4f55-ad3d-b57db6a68378 tempest-VolumeQuotaClassesTest-1774721437 tempest-VolumeQuotaClassesTest-1774721437-project-admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:52.921406 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627bb68f-9ac7-4f55-ad3d-b57db6a68378 tempest-VolumeQuotaClassesTest-1774721437 tempest-VolumeQuotaClassesTest-1774721437-project-admin] PATH_INFO: `/v3/projects/5528008b1504469c9c3b82aca9ce67a2` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:52.922521 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627bb68f-9ac7-4f55-ad3d-b57db6a68378 tempest-VolumeQuotaClassesTest-1774721437 tempest-VolumeQuotaClassesTest-1774721437-project-admin] RBAC: Authorizing `identity:delete_project(project_id=5528008b1504469c9c3b82aca9ce67a2)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:52.924521 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627bb68f-9ac7-4f55-ad3d-b57db6a68378 tempest-VolumeQuotaClassesTest-1774721437 tempest-VolumeQuotaClassesTest-1774721437-project-admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:52.952904 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:52.991618 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-627bb68f-9ac7-4f55-ad3d-b57db6a68378 tempest-VolumeQuotaClassesTest-1774721437 tempest-VolumeQuotaClassesTest-1774721437-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5528008b1504469c9c3b82aca9ce67a2', 'request_id': 'req-627bb68f-9ac7-4f55-ad3d-b57db6a68378'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:52.999596 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-627bb68f-9ac7-4f55-ad3d-b57db6a68378 tempest-VolumeQuotaClassesTest-1774721437 tempest-VolumeQuotaClassesTest-1774721437-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5528008b1504469c9c3b82aca9ce67a2', 'request_id': 'req-627bb68f-9ac7-4f55-ad3d-b57db6a68378'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:53.006997 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-627bb68f-9ac7-4f55-ad3d-b57db6a68378 tempest-VolumeQuotaClassesTest-1774721437 tempest-VolumeQuotaClassesTest-1774721437-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5528008b1504469c9c3b82aca9ce67a2', 'request_id': 'req-627bb68f-9ac7-4f55-ad3d-b57db6a68378'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:53.017467 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-627bb68f-9ac7-4f55-ad3d-b57db6a68378 tempest-VolumeQuotaClassesTest-1774721437 tempest-VolumeQuotaClassesTest-1774721437-project-admin] The token cache is being invalidate because project 5528008b1504469c9c3b82aca9ce67a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:14:53.017749 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-627bb68f-9ac7-4f55-ad3d-b57db6a68378 tempest-VolumeQuotaClassesTest-1774721437 tempest-VolumeQuotaClassesTest-1774721437-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:53.018384 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-627bb68f-9ac7-4f55-ad3d-b57db6a68378 tempest-VolumeQuotaClassesTest-1774721437 tempest-VolumeQuotaClassesTest-1774721437-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:53.019906 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 969/1938] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:14:52 2022] DELETE /identity/v3/projects/5528008b1504469c9c3b82aca9ce67a2 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 12:14:53.224625 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fee1c9-7c2d-400c-8e96-ad6eaaa4f537 tempest-VolumeQuotaClassesTest-1774721437 tempest-VolumeQuotaClassesTest-1774721437-project-admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:53.227214 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-f3fee1c9-7c2d-400c-8e96-ad6eaaa4f537 tempest-VolumeQuotaClassesTest-1774721437 tempest-VolumeQuotaClassesTest-1774721437-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:53.316243 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fee1c9-7c2d-400c-8e96-ad6eaaa4f537 tempest-VolumeQuotaClassesTest-1774721437 tempest-VolumeQuotaClassesTest-1774721437-project-admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:53.379708 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:53.400581 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5216da-cb3a-4ebe-a9ba-480aedca880b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:53.401430 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5216da-cb3a-4ebe-a9ba-480aedca880b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:53.401821 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5216da-cb3a-4ebe-a9ba-480aedca880b admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:53.402326 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5216da-cb3a-4ebe-a9ba-480aedca880b admin admin] PATH_INFO: `/v3/users/b525d07d26154a6aa3749328c080881f` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:53.403973 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5216da-cb3a-4ebe-a9ba-480aedca880b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b525d07d26154a6aa3749328c080881f)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:53.406161 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5216da-cb3a-4ebe-a9ba-480aedca880b admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:53.406501 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c507f-9319-485e-9edf-2e3a0bd82cc7 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:53.406791 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c507f-9319-485e-9edf-2e3a0bd82cc7 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:53.407066 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c507f-9319-485e-9edf-2e3a0bd82cc7 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:53.408606 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 970/1939] 10.222.0.61 () {54 vars in 921 bytes} [Fri Aug 26 12:14:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:14:53.419871 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010cb906-f09c-4d62-af2c-873b08dd16e8 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:53.420165 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010cb906-f09c-4d62-af2c-873b08dd16e8 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:53.420448 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010cb906-f09c-4d62-af2c-873b08dd16e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:53.450572 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-010cb906-f09c-4d62-af2c-873b08dd16e8 None None] MFA Rules not processed for user `7aacd7d6d17e4f10ba5c9107a12beafa`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:53.452681 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-010cb906-f09c-4d62-af2c-873b08dd16e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:53.498812 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-1e5216da-cb3a-4ebe-a9ba-480aedca880b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b525d07d26154a6aa3749328c080881f'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:53.509920 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-1e5216da-cb3a-4ebe-a9ba-480aedca880b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b525d07d26154a6aa3749328c080881f'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:53.510634 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-1e5216da-cb3a-4ebe-a9ba-480aedca880b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b525d07d26154a6aa3749328c080881f'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:53.520160 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-1e5216da-cb3a-4ebe-a9ba-480aedca880b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b525d07d26154a6aa3749328c080881f'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:53.520829 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-1e5216da-cb3a-4ebe-a9ba-480aedca880b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b525d07d26154a6aa3749328c080881f'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:53.534083 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 971/1940] 10.222.0.61 () {58 vars in 1018 bytes} [Fri Aug 26 12:14:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:53.547114 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010cb906-f09c-4d62-af2c-873b08dd16e8 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:53.547632 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 970/1941] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:14:53 2022] DELETE /identity/v3/users/b525d07d26154a6aa3749328c080881f => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:53.551841 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be44a68-2752-4d09-8d88-82205cd15bec None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:53.552126 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be44a68-2752-4d09-8d88-82205cd15bec None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:53.552428 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be44a68-2752-4d09-8d88-82205cd15bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:53.556845 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-010cb906-f09c-4d62-af2c-873b08dd16e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:53.578529 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010cb906-f09c-4d62-af2c-873b08dd16e8 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:53.585905 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-5be44a68-2752-4d09-8d88-82205cd15bec None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:53.587476 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-5be44a68-2752-4d09-8d88-82205cd15bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:53.680148 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad28001b-9c05-4a65-9c48-7e3ff261353b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aacd7d6d17e4f10ba5c9107a12beafa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:53.680400 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad28001b-9c05-4a65-9c48-7e3ff261353b service nova] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:53.681160 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad28001b-9c05-4a65-9c48-7e3ff261353b service nova] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:53.681160 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad28001b-9c05-4a65-9c48-7e3ff261353b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:53.698052 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad28001b-9c05-4a65-9c48-7e3ff261353b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:53.700788 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad28001b-9c05-4a65-9c48-7e3ff261353b service nova] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:53.722246 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 972/1942] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:14:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 12:14:53.738662 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 971/1943] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:14:53.751582 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad28001b-9c05-4a65-9c48-7e3ff261353b service nova] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:53.754518 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-ad28001b-9c05-4a65-9c48-7e3ff261353b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:53.804163 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:53.855323 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad28001b-9c05-4a65-9c48-7e3ff261353b service nova] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:53.872039 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa20d2c-5267-4d8e-aac1-7313acd1e958 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:53.872039 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa20d2c-5267-4d8e-aac1-7313acd1e958 service neutron] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:53.872039 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa20d2c-5267-4d8e-aac1-7313acd1e958 service neutron] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:53.872039 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa20d2c-5267-4d8e-aac1-7313acd1e958 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:53.891482 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa20d2c-5267-4d8e-aac1-7313acd1e958 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:53.895623 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa20d2c-5267-4d8e-aac1-7313acd1e958 service neutron] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:53.917861 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 973/1944] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:14:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:14:53.961415 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be44a68-2752-4d09-8d88-82205cd15bec None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:53.963795 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-5be44a68-2752-4d09-8d88-82205cd15bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:54.061398 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be44a68-2752-4d09-8d88-82205cd15bec None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:54.065527 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36dc6c0-fdbd-4854-b83e-34617ba0712a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:54.066319 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36dc6c0-fdbd-4854-b83e-34617ba0712a service cinder] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:54.067011 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36dc6c0-fdbd-4854-b83e-34617ba0712a service cinder] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:54.067518 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36dc6c0-fdbd-4854-b83e-34617ba0712a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:54.069719 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-e36dc6c0-fdbd-4854-b83e-34617ba0712a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:54.094316 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:54.140726 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa20d2c-5267-4d8e-aac1-7313acd1e958 service neutron] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:54.142720 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-cfa20d2c-5267-4d8e-aac1-7313acd1e958 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:54.156618 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36dc6c0-fdbd-4854-b83e-34617ba0712a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:54.158204 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36dc6c0-fdbd-4854-b83e-34617ba0712a service cinder] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:54.167316 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa20d2c-5267-4d8e-aac1-7313acd1e958 service neutron] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:54.175035 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8dfa5d-cec2-4498-ab63-a351b3412d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:54.175851 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8dfa5d-cec2-4498-ab63-a351b3412d55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:54.176348 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8dfa5d-cec2-4498-ab63-a351b3412d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:54.176728 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8dfa5d-cec2-4498-ab63-a351b3412d55 admin admin] PATH_INFO: `/v3/users/e5e0d43e61894d1c9353178a128190e7` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:54.177290 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 972/1945] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:14:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:14:54.189712 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8dfa5d-cec2-4498-ab63-a351b3412d55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5e0d43e61894d1c9353178a128190e7)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:54.193053 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8dfa5d-cec2-4498-ab63-a351b3412d55 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:54.268362 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-dc8dfa5d-cec2-4498-ab63-a351b3412d55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5e0d43e61894d1c9353178a128190e7'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:54.280551 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-dc8dfa5d-cec2-4498-ab63-a351b3412d55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5e0d43e61894d1c9353178a128190e7'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:54.288820 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-dc8dfa5d-cec2-4498-ab63-a351b3412d55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5e0d43e61894d1c9353178a128190e7'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:54.289630 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-dc8dfa5d-cec2-4498-ab63-a351b3412d55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5e0d43e61894d1c9353178a128190e7'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:54.306047 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-dc8dfa5d-cec2-4498-ab63-a351b3412d55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5e0d43e61894d1c9353178a128190e7'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:54.314554 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 974/1946] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:14:54 2022] DELETE /identity/v3/users/e5e0d43e61894d1c9353178a128190e7 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 12:14:54.319793 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449c021-6be2-48bf-b67d-d5dd79eb8ad1 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:54.320961 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449c021-6be2-48bf-b67d-d5dd79eb8ad1 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:54.321052 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449c021-6be2-48bf-b67d-d5dd79eb8ad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:54.340907 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-2449c021-6be2-48bf-b67d-d5dd79eb8ad1 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:54.342136 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-2449c021-6be2-48bf-b67d-d5dd79eb8ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:54.440303 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 973/1947] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:54.450315 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8dfa5d-cec2-4498-ab63-a351b3412d55 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:54.452099 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-dc8dfa5d-cec2-4498-ab63-a351b3412d55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:54.502492 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2449c021-6be2-48bf-b67d-d5dd79eb8ad1 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:54.505002 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-2449c021-6be2-48bf-b67d-d5dd79eb8ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:54.526307 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2449c021-6be2-48bf-b67d-d5dd79eb8ad1 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:54.527156 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8dfa5d-cec2-4498-ab63-a351b3412d55 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:54.532285 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a92d09b-9784-4012-87d6-0529e1a13427 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:54.533247 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92d09b-9784-4012-87d6-0529e1a13427 service neutron] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:54.533849 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92d09b-9784-4012-87d6-0529e1a13427 service neutron] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:54.534289 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92d09b-9784-4012-87d6-0529e1a13427 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:54.538099 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c707fbf1-0b7e-4d6f-98d3-80a85b5df5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:54.539750 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c707fbf1-0b7e-4d6f-98d3-80a85b5df5d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:54.540352 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c707fbf1-0b7e-4d6f-98d3-80a85b5df5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:54.540780 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c707fbf1-0b7e-4d6f-98d3-80a85b5df5d8 admin admin] PATH_INFO: `/v3/projects/883e9e5cec154ac79fdeab55e2ddf282` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:54.542329 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c707fbf1-0b7e-4d6f-98d3-80a85b5df5d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=883e9e5cec154ac79fdeab55e2ddf282)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:54.545319 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c707fbf1-0b7e-4d6f-98d3-80a85b5df5d8 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:54.545985 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a92d09b-9784-4012-87d6-0529e1a13427 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:54.548151 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a92d09b-9784-4012-87d6-0529e1a13427 service neutron] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:54.567380 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 975/1948] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:14:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:14:54.601675 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-c707fbf1-0b7e-4d6f-98d3-80a85b5df5d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '883e9e5cec154ac79fdeab55e2ddf282', 'request_id': 'req-c707fbf1-0b7e-4d6f-98d3-80a85b5df5d8'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:54.614184 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-c707fbf1-0b7e-4d6f-98d3-80a85b5df5d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '883e9e5cec154ac79fdeab55e2ddf282', 'request_id': 'req-c707fbf1-0b7e-4d6f-98d3-80a85b5df5d8'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:54.624155 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-c707fbf1-0b7e-4d6f-98d3-80a85b5df5d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '883e9e5cec154ac79fdeab55e2ddf282', 'request_id': 'req-c707fbf1-0b7e-4d6f-98d3-80a85b5df5d8'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:54.663082 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-c707fbf1-0b7e-4d6f-98d3-80a85b5df5d8 admin admin] The token cache is being invalidate because project 883e9e5cec154ac79fdeab55e2ddf282 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81712) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:14:54.663474 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-c707fbf1-0b7e-4d6f-98d3-80a85b5df5d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:54.667955 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-c707fbf1-0b7e-4d6f-98d3-80a85b5df5d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:54.669744 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 974/1949] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:14:54 2022] DELETE /identity/v3/projects/883e9e5cec154ac79fdeab55e2ddf282 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:54.674513 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a92d09b-9784-4012-87d6-0529e1a13427 service neutron] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:54.686547 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a92d09b-9784-4012-87d6-0529e1a13427 service neutron] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:54.782668 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:54.814398 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91274c7b-a6b3-4211-8da8-bad4d0e39235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:54.815983 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91274c7b-a6b3-4211-8da8-bad4d0e39235 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:54.816374 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91274c7b-a6b3-4211-8da8-bad4d0e39235 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:54.816748 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91274c7b-a6b3-4211-8da8-bad4d0e39235 admin admin] PATH_INFO: `/v3/users/2e71e248c58b446186399dcdb0a03ee0` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:54.828011 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91274c7b-a6b3-4211-8da8-bad4d0e39235 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e71e248c58b446186399dcdb0a03ee0)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:54.836706 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91274c7b-a6b3-4211-8da8-bad4d0e39235 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:54.924135 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-91274c7b-a6b3-4211-8da8-bad4d0e39235 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e71e248c58b446186399dcdb0a03ee0'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:54.937169 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-91274c7b-a6b3-4211-8da8-bad4d0e39235 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e71e248c58b446186399dcdb0a03ee0'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:54.949003 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-91274c7b-a6b3-4211-8da8-bad4d0e39235 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e71e248c58b446186399dcdb0a03ee0'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:54.950018 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-91274c7b-a6b3-4211-8da8-bad4d0e39235 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e71e248c58b446186399dcdb0a03ee0'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:54.955140 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c707fbf1-0b7e-4d6f-98d3-80a85b5df5d8 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:54.959409 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-c707fbf1-0b7e-4d6f-98d3-80a85b5df5d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:54.978188 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-91274c7b-a6b3-4211-8da8-bad4d0e39235 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e71e248c58b446186399dcdb0a03ee0'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:54.986522 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 976/1950] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:14:54 2022] DELETE /identity/v3/users/2e71e248c58b446186399dcdb0a03ee0 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:55.033727 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:55.050169 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c707fbf1-0b7e-4d6f-98d3-80a85b5df5d8 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:55.139810 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96c14b6-bd59-40a0-b121-26fdd8b74d10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:55.140953 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c14b6-bd59-40a0-b121-26fdd8b74d10 service cinder] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:55.141451 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c14b6-bd59-40a0-b121-26fdd8b74d10 service cinder] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:55.141928 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c14b6-bd59-40a0-b121-26fdd8b74d10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:55.145030 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-e96c14b6-bd59-40a0-b121-26fdd8b74d10 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:55.339158 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96c14b6-bd59-40a0-b121-26fdd8b74d10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:55.340846 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96c14b6-bd59-40a0-b121-26fdd8b74d10 service cinder] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:55.364364 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 975/1951] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:14:54 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:14:56.204080 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91274c7b-a6b3-4211-8da8-bad4d0e39235 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:56.206265 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-91274c7b-a6b3-4211-8da8-bad4d0e39235 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:56.310353 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91274c7b-a6b3-4211-8da8-bad4d0e39235 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:56.318205 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36099534-c2ca-4db0-a999-37a1f81a8a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:56.318610 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36099534-c2ca-4db0-a999-37a1f81a8a37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:56.318898 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36099534-c2ca-4db0-a999-37a1f81a8a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:56.319174 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36099534-c2ca-4db0-a999-37a1f81a8a37 admin admin] PATH_INFO: `/v3/projects/5a903596b7d247648c39ebc48695ac86` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:56.321767 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36099534-c2ca-4db0-a999-37a1f81a8a37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a903596b7d247648c39ebc48695ac86)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:56.325000 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36099534-c2ca-4db0-a999-37a1f81a8a37 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:56.343873 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96c14b6-bd59-40a0-b121-26fdd8b74d10 service cinder] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:56.350412 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-e96c14b6-bd59-40a0-b121-26fdd8b74d10 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:56.372243 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96c14b6-bd59-40a0-b121-26fdd8b74d10 service cinder] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:56.377741 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d136dc0-797f-4a83-956d-9df58d4c3d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:56.378917 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d136dc0-797f-4a83-956d-9df58d4c3d69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:56.379414 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d136dc0-797f-4a83-956d-9df58d4c3d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:56.379840 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d136dc0-797f-4a83-956d-9df58d4c3d69 admin admin] PATH_INFO: `/v3/projects/6f83ad913d9543d3a7d06665590c1998` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:56.381112 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d136dc0-797f-4a83-956d-9df58d4c3d69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f83ad913d9543d3a7d06665590c1998)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:56.383680 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d136dc0-797f-4a83-956d-9df58d4c3d69 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:56.426305 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-36099534-c2ca-4db0-a999-37a1f81a8a37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a903596b7d247648c39ebc48695ac86', 'request_id': 'req-36099534-c2ca-4db0-a999-37a1f81a8a37'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:56.441172 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-36099534-c2ca-4db0-a999-37a1f81a8a37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a903596b7d247648c39ebc48695ac86', 'request_id': 'req-36099534-c2ca-4db0-a999-37a1f81a8a37'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:56.466120 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-36099534-c2ca-4db0-a999-37a1f81a8a37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a903596b7d247648c39ebc48695ac86', 'request_id': 'req-36099534-c2ca-4db0-a999-37a1f81a8a37'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:56.468835 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:56.484141 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-36099534-c2ca-4db0-a999-37a1f81a8a37 admin admin] The token cache is being invalidate because project 5a903596b7d247648c39ebc48695ac86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:14:56.484141 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-36099534-c2ca-4db0-a999-37a1f81a8a37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:56.485929 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-36099534-c2ca-4db0-a999-37a1f81a8a37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:56.489228 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-7d136dc0-797f-4a83-956d-9df58d4c3d69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f83ad913d9543d3a7d06665590c1998', 'request_id': 'req-7d136dc0-797f-4a83-956d-9df58d4c3d69'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:56.494234 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 977/1952] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:14:56 2022] DELETE /identity/v3/projects/5a903596b7d247648c39ebc48695ac86 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:56.497853 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-7d136dc0-797f-4a83-956d-9df58d4c3d69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f83ad913d9543d3a7d06665590c1998', 'request_id': 'req-7d136dc0-797f-4a83-956d-9df58d4c3d69'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:56.498213 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36099534-c2ca-4db0-a999-37a1f81a8a37 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:56.499637 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-36099534-c2ca-4db0-a999-37a1f81a8a37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:56.512475 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-7d136dc0-797f-4a83-956d-9df58d4c3d69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f83ad913d9543d3a7d06665590c1998', 'request_id': 'req-7d136dc0-797f-4a83-956d-9df58d4c3d69'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:56.540553 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-7d136dc0-797f-4a83-956d-9df58d4c3d69 admin admin] The token cache is being invalidate because project 6f83ad913d9543d3a7d06665590c1998 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81712) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:14:56.541068 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-7d136dc0-797f-4a83-956d-9df58d4c3d69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:56.542104 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-7d136dc0-797f-4a83-956d-9df58d4c3d69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:56.548675 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 976/1953] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:14:56 2022] DELETE /identity/v3/projects/6f83ad913d9543d3a7d06665590c1998 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:56.550557 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb94cb30-8724-4f5e-b1fa-11828595fb25 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:56.550979 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb94cb30-8724-4f5e-b1fa-11828595fb25 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:56.551280 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb94cb30-8724-4f5e-b1fa-11828595fb25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:56.582012 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-bb94cb30-8724-4f5e-b1fa-11828595fb25 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:56.583444 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-bb94cb30-8724-4f5e-b1fa-11828595fb25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:56.630728 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:56.654243 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36099534-c2ca-4db0-a999-37a1f81a8a37 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:56.679120 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 977/1954] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:56.688997 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c72b8-4fc9-415f-a147-b248fb9e201c None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:56.689279 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c72b8-4fc9-415f-a147-b248fb9e201c None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:56.689551 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c72b8-4fc9-415f-a147-b248fb9e201c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:56.712112 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-ab8c72b8-4fc9-415f-a147-b248fb9e201c None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:56.713376 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-ab8c72b8-4fc9-415f-a147-b248fb9e201c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:56.722108 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 978/1955] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:56.728032 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8c72b8-4fc9-415f-a147-b248fb9e201c None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:56.742622 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8c72b8-4fc9-415f-a147-b248fb9e201c None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:56.749927 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7a2c3d-a028-4ac3-b7c9-cfbc8a374bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:56.750873 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7a2c3d-a028-4ac3-b7c9-cfbc8a374bed admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:56.753733 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7a2c3d-a028-4ac3-b7c9-cfbc8a374bed admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:56.754127 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7a2c3d-a028-4ac3-b7c9-cfbc8a374bed admin admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:56.754959 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7a2c3d-a028-4ac3-b7c9-cfbc8a374bed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:14:56.755378 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7a2c3d-a028-4ac3-b7c9-cfbc8a374bed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:56.757282 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7a2c3d-a028-4ac3-b7c9-cfbc8a374bed admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:56.761422 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1421dad5-05a0-4cef-9523-540b37e7b1cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:56.762653 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1421dad5-05a0-4cef-9523-540b37e7b1cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:56.762939 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1421dad5-05a0-4cef-9523-540b37e7b1cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:56.763359 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1421dad5-05a0-4cef-9523-540b37e7b1cc admin admin] PATH_INFO: `/v3/users/ff21489c72c649f68e30ea0ece1f2fca` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:56.764570 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1421dad5-05a0-4cef-9523-540b37e7b1cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff21489c72c649f68e30ea0ece1f2fca)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:56.773384 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1421dad5-05a0-4cef-9523-540b37e7b1cc admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:56.782680 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 979/1956] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:14:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:14:56.783638 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7a2c3d-a028-4ac3-b7c9-cfbc8a374bed admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:56.797230 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7a2c3d-a028-4ac3-b7c9-cfbc8a374bed admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:56.802409 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eddd2d-3fc0-4d52-a672-bd41082fc733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:56.803176 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eddd2d-3fc0-4d52-a672-bd41082fc733 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:56.803510 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eddd2d-3fc0-4d52-a672-bd41082fc733 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:56.803791 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eddd2d-3fc0-4d52-a672-bd41082fc733 admin admin] PATH_INFO: `/v3/projects` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:56.805271 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eddd2d-3fc0-4d52-a672-bd41082fc733 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:56.807593 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eddd2d-3fc0-4d52-a672-bd41082fc733 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:56.846971 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-1421dad5-05a0-4cef-9523-540b37e7b1cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff21489c72c649f68e30ea0ece1f2fca'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:56.853784 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 980/1957] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:14:56 2022] POST /identity/v3/projects => generated 381 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:14:56.855232 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eddd2d-3fc0-4d52-a672-bd41082fc733 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:56.860048 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-1421dad5-05a0-4cef-9523-540b37e7b1cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff21489c72c649f68e30ea0ece1f2fca'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:56.871324 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-1421dad5-05a0-4cef-9523-540b37e7b1cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff21489c72c649f68e30ea0ece1f2fca'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:56.872386 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-1421dad5-05a0-4cef-9523-540b37e7b1cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff21489c72c649f68e30ea0ece1f2fca'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:56.899183 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-1421dad5-05a0-4cef-9523-540b37e7b1cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff21489c72c649f68e30ea0ece1f2fca'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:56.908425 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 978/1958] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:14:56 2022] DELETE /identity/v3/users/ff21489c72c649f68e30ea0ece1f2fca => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 12:14:56.988376 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eddd2d-3fc0-4d52-a672-bd41082fc733 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:57.081459 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a593d2-9a81-4e0f-b4c3-3ee71bf4954c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:57.081660 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a593d2-9a81-4e0f-b4c3-3ee71bf4954c admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:57.081970 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a593d2-9a81-4e0f-b4c3-3ee71bf4954c admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:57.082220 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a593d2-9a81-4e0f-b4c3-3ee71bf4954c admin admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:57.082956 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a593d2-9a81-4e0f-b4c3-3ee71bf4954c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:57.085823 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a593d2-9a81-4e0f-b4c3-3ee71bf4954c admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:57.120257 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 981/1959] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:14:56 2022] POST /identity/v3/users => generated 291 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:14:57.124170 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1421dad5-05a0-4cef-9523-540b37e7b1cc admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:57.125598 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-1421dad5-05a0-4cef-9523-540b37e7b1cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:57.144858 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1421dad5-05a0-4cef-9523-540b37e7b1cc admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:57.159227 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bcea59-cc3c-4a12-bc29-86d9d777d0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:57.159227 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bcea59-cc3c-4a12-bc29-86d9d777d0a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:57.159227 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bcea59-cc3c-4a12-bc29-86d9d777d0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:57.159227 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bcea59-cc3c-4a12-bc29-86d9d777d0a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:57.159609 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bcea59-cc3c-4a12-bc29-86d9d777d0a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:57.159609 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bcea59-cc3c-4a12-bc29-86d9d777d0a7 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:57.165934 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 979/1960] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:14:57 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:14:57.170032 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a593d2-9a81-4e0f-b4c3-3ee71bf4954c admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:57.180629 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a593d2-9a81-4e0f-b4c3-3ee71bf4954c admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:57.185187 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5270c6bc-5869-4343-987c-5cf3e8e9da45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:57.185780 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5270c6bc-5869-4343-987c-5cf3e8e9da45 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:57.186060 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5270c6bc-5869-4343-987c-5cf3e8e9da45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:57.186336 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5270c6bc-5869-4343-987c-5cf3e8e9da45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:57.186973 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5270c6bc-5869-4343-987c-5cf3e8e9da45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:57.188606 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5270c6bc-5869-4343-987c-5cf3e8e9da45 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:57.199394 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 982/1961] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:14:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:14:57.204404 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bcea59-cc3c-4a12-bc29-86d9d777d0a7 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:57.218372 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bcea59-cc3c-4a12-bc29-86d9d777d0a7 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:57.222991 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3635d2b5-6c1e-4308-a1e6-46f4138679a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:57.223677 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3635d2b5-6c1e-4308-a1e6-46f4138679a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:57.223955 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3635d2b5-6c1e-4308-a1e6-46f4138679a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:57.224222 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3635d2b5-6c1e-4308-a1e6-46f4138679a1 admin admin] PATH_INFO: `/v3/projects/ceb6d267cb364989a4767c22eed4defd/users/b0f5701503774a03ab9ca24982c18d35/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:57.227094 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:57.243240 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3635d2b5-6c1e-4308-a1e6-46f4138679a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ceb6d267cb364989a4767c22eed4defd, user_id=b0f5701503774a03ab9ca24982c18d35, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:57.245815 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3635d2b5-6c1e-4308-a1e6-46f4138679a1 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:57.260234 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 980/1962] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:14:57 2022] PUT /identity/v3/projects/ceb6d267cb364989a4767c22eed4defd/users/b0f5701503774a03ab9ca24982c18d35/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:57.270269 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe00f9-c0e7-40ca-99ad-3b548283bc80 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:57.270559 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe00f9-c0e7-40ca-99ad-3b548283bc80 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:57.270830 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe00f9-c0e7-40ca-99ad-3b548283bc80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:57.316119 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-44fe00f9-c0e7-40ca-99ad-3b548283bc80 None None] MFA Rules not processed for user `b0f5701503774a03ab9ca24982c18d35`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:57.317517 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-44fe00f9-c0e7-40ca-99ad-3b548283bc80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:57.325533 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:57.452412 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3635d2b5-6c1e-4308-a1e6-46f4138679a1 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:57.458217 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-3635d2b5-6c1e-4308-a1e6-46f4138679a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:57.458752 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 983/1963] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:57 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:57.463829 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fe00f9-c0e7-40ca-99ad-3b548283bc80 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:57.552496 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fe00f9-c0e7-40ca-99ad-3b548283bc80 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:57.558561 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442cdb79-ba50-4f85-96e2-3f989a9afbd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:57.560063 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442cdb79-ba50-4f85-96e2-3f989a9afbd8 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:57.560172 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442cdb79-ba50-4f85-96e2-3f989a9afbd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:57.560439 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442cdb79-ba50-4f85-96e2-3f989a9afbd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:57.561080 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442cdb79-ba50-4f85-96e2-3f989a9afbd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:57.562993 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442cdb79-ba50-4f85-96e2-3f989a9afbd8 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:57.563721 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3635d2b5-6c1e-4308-a1e6-46f4138679a1 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:57.584094 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75954092-4650-45f7-a602-1cd0b290452f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:57.584765 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75954092-4650-45f7-a602-1cd0b290452f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:57.585048 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75954092-4650-45f7-a602-1cd0b290452f admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:57.585388 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75954092-4650-45f7-a602-1cd0b290452f admin admin] PATH_INFO: `/v3/projects/bd63eb2f3f5c4752a3edc3c9b2e63c53` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:57.590832 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75954092-4650-45f7-a602-1cd0b290452f admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd63eb2f3f5c4752a3edc3c9b2e63c53)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:57.593351 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75954092-4650-45f7-a602-1cd0b290452f admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:57.626956 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 984/1964] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:14:57 2022] POST /identity/v3/projects => generated 381 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 12:14:57.627973 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442cdb79-ba50-4f85-96e2-3f989a9afbd8 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:57.685560 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-75954092-4650-45f7-a602-1cd0b290452f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd63eb2f3f5c4752a3edc3c9b2e63c53', 'request_id': 'req-75954092-4650-45f7-a602-1cd0b290452f'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:57.695383 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-75954092-4650-45f7-a602-1cd0b290452f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd63eb2f3f5c4752a3edc3c9b2e63c53', 'request_id': 'req-75954092-4650-45f7-a602-1cd0b290452f'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:57.708902 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-75954092-4650-45f7-a602-1cd0b290452f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd63eb2f3f5c4752a3edc3c9b2e63c53', 'request_id': 'req-75954092-4650-45f7-a602-1cd0b290452f'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:57.719555 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442cdb79-ba50-4f85-96e2-3f989a9afbd8 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:57.725170 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-75954092-4650-45f7-a602-1cd0b290452f admin admin] The token cache is being invalidate because project bd63eb2f3f5c4752a3edc3c9b2e63c53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:14:57.725808 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-75954092-4650-45f7-a602-1cd0b290452f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:57.727177 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-75954092-4650-45f7-a602-1cd0b290452f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:14:57.728919 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 981/1965] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:14:57 2022] DELETE /identity/v3/projects/bd63eb2f3f5c4752a3edc3c9b2e63c53 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 12:14:57.733392 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e680695-0e75-4a3c-a65e-b66f9fb041c5 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:57.733832 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e680695-0e75-4a3c-a65e-b66f9fb041c5 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:57.734276 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e680695-0e75-4a3c-a65e-b66f9fb041c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:57.759472 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-8e680695-0e75-4a3c-a65e-b66f9fb041c5 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:57.760694 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-8e680695-0e75-4a3c-a65e-b66f9fb041c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:57.763612 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:57.793383 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bc11db-4fd4-4f5f-a0be-2cda5a8791cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:57.794201 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bc11db-4fd4-4f5f-a0be-2cda5a8791cf admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:57.794925 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bc11db-4fd4-4f5f-a0be-2cda5a8791cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:57.794925 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bc11db-4fd4-4f5f-a0be-2cda5a8791cf admin admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:57.795592 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bc11db-4fd4-4f5f-a0be-2cda5a8791cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:57.797572 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bc11db-4fd4-4f5f-a0be-2cda5a8791cf admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:57.820610 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 985/1966] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:14:57 2022] POST /identity/v3/users => generated 297 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:14:57.824791 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bc11db-4fd4-4f5f-a0be-2cda5a8791cf admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:57.826743 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-d1bc11db-4fd4-4f5f-a0be-2cda5a8791cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:57.834110 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 982/1967] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:14:57.841663 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bc11db-4fd4-4f5f-a0be-2cda5a8791cf admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:57.843386 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a793f62-4509-465e-89c2-70d76a353714 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:57.843680 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a793f62-4509-465e-89c2-70d76a353714 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:57.843951 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a793f62-4509-465e-89c2-70d76a353714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:57.847563 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98e851a-b398-4543-8d36-0ba7ef83cf51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:57.847835 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98e851a-b398-4543-8d36-0ba7ef83cf51 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:57.848381 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98e851a-b398-4543-8d36-0ba7ef83cf51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:57.848648 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98e851a-b398-4543-8d36-0ba7ef83cf51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:57.849362 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98e851a-b398-4543-8d36-0ba7ef83cf51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:57.851891 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98e851a-b398-4543-8d36-0ba7ef83cf51 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:57.868354 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 986/1968] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:14:57 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 12:14:57.869437 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98e851a-b398-4543-8d36-0ba7ef83cf51 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:57.869727 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-6a793f62-4509-465e-89c2-70d76a353714 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:57.870992 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-6a793f62-4509-465e-89c2-70d76a353714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:57.879014 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 983/1969] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:14:57.880157 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98e851a-b398-4543-8d36-0ba7ef83cf51 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:57.883087 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a793f62-4509-465e-89c2-70d76a353714 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:57.886243 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dd2146-63db-42dc-a611-404effe46f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:57.887256 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd2146-63db-42dc-a611-404effe46f92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:57.887586 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd2146-63db-42dc-a611-404effe46f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:57.887861 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd2146-63db-42dc-a611-404effe46f92 admin admin] PATH_INFO: `/v3/projects/ab07e470aded4662afbf69833ce7932d/users/c788c31040454631a2f76aa1564a3650/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:57.893775 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a793f62-4509-465e-89c2-70d76a353714 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:57.899300 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1ca7b4-baa1-4f91-846b-f9d51eed69f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:57.899998 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ca7b4-baa1-4f91-846b-f9d51eed69f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:57.900350 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ca7b4-baa1-4f91-846b-f9d51eed69f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:57.900944 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ca7b4-baa1-4f91-846b-f9d51eed69f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:57.901405 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1ca7b4-baa1-4f91-846b-f9d51eed69f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:14:57.901790 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1ca7b4-baa1-4f91-846b-f9d51eed69f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:57.903516 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1ca7b4-baa1-4f91-846b-f9d51eed69f2 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:57.905859 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:57.909065 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dd2146-63db-42dc-a611-404effe46f92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab07e470aded4662afbf69833ce7932d, user_id=c788c31040454631a2f76aa1564a3650, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:57.912447 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dd2146-63db-42dc-a611-404effe46f92 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:57.923618 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 984/1970] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:14:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:14:57.924630 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1ca7b4-baa1-4f91-846b-f9d51eed69f2 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:57.934749 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 987/1971] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:14:57 2022] PUT /identity/v3/projects/ab07e470aded4662afbf69833ce7932d/users/c788c31040454631a2f76aa1564a3650/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:57.938418 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1ca7b4-baa1-4f91-846b-f9d51eed69f2 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:57.938681 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dd2146-63db-42dc-a611-404effe46f92 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:58.031412 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1f595b-6819-4e17-b43e-f33e7e312d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:58.032541 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f595b-6819-4e17-b43e-f33e7e312d72 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:58.033125 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f595b-6819-4e17-b43e-f33e7e312d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:58.033819 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f595b-6819-4e17-b43e-f33e7e312d72 admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:58.034627 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1f595b-6819-4e17-b43e-f33e7e312d72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:58.038225 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1f595b-6819-4e17-b43e-f33e7e312d72 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:58.104738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 985/1972] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:14:57 2022] POST /identity/v3/projects => generated 373 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:14:58.105647 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1f595b-6819-4e17-b43e-f33e7e312d72 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:58.124908 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dd2146-63db-42dc-a611-404effe46f92 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:58.205173 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1f595b-6819-4e17-b43e-f33e7e312d72 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:58.211326 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcb9fba-dd30-4722-b7db-72cacbac3fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:58.212067 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb9fba-dd30-4722-b7db-72cacbac3fd6 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:58.212351 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb9fba-dd30-4722-b7db-72cacbac3fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:58.212682 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb9fba-dd30-4722-b7db-72cacbac3fd6 admin admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:58.214120 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcb9fba-dd30-4722-b7db-72cacbac3fd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:58.218844 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcb9fba-dd30-4722-b7db-72cacbac3fd6 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:58.255060 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 986/1973] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:14:58 2022] POST /identity/v3/users => generated 287 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:14:58.260566 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcb9fba-dd30-4722-b7db-72cacbac3fd6 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:58.273181 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcb9fba-dd30-4722-b7db-72cacbac3fd6 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:58.280020 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84749477-ad2a-4000-a056-697ef67b4967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:58.280833 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84749477-ad2a-4000-a056-697ef67b4967 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:58.281112 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84749477-ad2a-4000-a056-697ef67b4967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:58.281380 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84749477-ad2a-4000-a056-697ef67b4967 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:58.281802 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5eba57-5f51-404a-8822-6ccdd2fa7bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:58.282071 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84749477-ad2a-4000-a056-697ef67b4967 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:58.282425 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5eba57-5f51-404a-8822-6ccdd2fa7bbb admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:58.282704 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5eba57-5f51-404a-8822-6ccdd2fa7bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:58.282973 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5eba57-5f51-404a-8822-6ccdd2fa7bbb admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:58.283654 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5eba57-5f51-404a-8822-6ccdd2fa7bbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:58.284109 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84749477-ad2a-4000-a056-697ef67b4967 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:58.285218 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5eba57-5f51-404a-8822-6ccdd2fa7bbb admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:58.302092 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 987/1974] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:14:58 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:14:58.303047 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84749477-ad2a-4000-a056-697ef67b4967 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:58.316110 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 988/1975] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:14:57 2022] GET /identity/v3/roles => generated 1420 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 12:14:58.316306 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84749477-ad2a-4000-a056-697ef67b4967 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:58.317240 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5eba57-5f51-404a-8822-6ccdd2fa7bbb admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:58.321553 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3227166-7cef-43a5-bb2f-9c32a50ba9bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:58.322776 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3227166-7cef-43a5-bb2f-9c32a50ba9bd admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:58.323170 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3227166-7cef-43a5-bb2f-9c32a50ba9bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:58.323549 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3227166-7cef-43a5-bb2f-9c32a50ba9bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:58.324248 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3227166-7cef-43a5-bb2f-9c32a50ba9bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:58.327741 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3227166-7cef-43a5-bb2f-9c32a50ba9bd admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:58.339665 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5eba57-5f51-404a-8822-6ccdd2fa7bbb admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:58.340888 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 988/1976] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:14:58 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:14:58.346288 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3227166-7cef-43a5-bb2f-9c32a50ba9bd admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:58.358894 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6badd943-56f5-41e5-bdd6-86f65c68ac14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:58.359889 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6badd943-56f5-41e5-bdd6-86f65c68ac14 admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:58.360003 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3227166-7cef-43a5-bb2f-9c32a50ba9bd admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:58.361500 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6badd943-56f5-41e5-bdd6-86f65c68ac14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:58.362074 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6badd943-56f5-41e5-bdd6-86f65c68ac14 admin admin] PATH_INFO: `/v3/projects/ab07e470aded4662afbf69833ce7932d/users/c788c31040454631a2f76aa1564a3650/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:58.367667 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421e5dcb-a8cb-497c-9a70-3537303b83b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:58.368753 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e5dcb-a8cb-497c-9a70-3537303b83b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:58.369347 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e5dcb-a8cb-497c-9a70-3537303b83b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:58.369459 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6badd943-56f5-41e5-bdd6-86f65c68ac14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab07e470aded4662afbf69833ce7932d, user_id=c788c31040454631a2f76aa1564a3650, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:58.369691 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e5dcb-a8cb-497c-9a70-3537303b83b8 admin admin] PATH_INFO: `/v3/projects/afc067f6c1e741d5a04b278d67dba2a6/users/0d56e72c69434537ad0a754492f5cd35/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:58.376946 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6badd943-56f5-41e5-bdd6-86f65c68ac14 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:58.403697 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 989/1977] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:14:58 2022] PUT /identity/v3/projects/ab07e470aded4662afbf69833ce7932d/users/c788c31040454631a2f76aa1564a3650/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:58.406095 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde84d0-5091-4a4f-abe6-317f2034cf8a None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:58.406389 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde84d0-5091-4a4f-abe6-317f2034cf8a None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:58.406939 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde84d0-5091-4a4f-abe6-317f2034cf8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:58.407631 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421e5dcb-a8cb-497c-9a70-3537303b83b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afc067f6c1e741d5a04b278d67dba2a6, user_id=0d56e72c69434537ad0a754492f5cd35, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:58.410493 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421e5dcb-a8cb-497c-9a70-3537303b83b8 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:58.451190 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 989/1978] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:14:58 2022] PUT /identity/v3/projects/afc067f6c1e741d5a04b278d67dba2a6/users/0d56e72c69434537ad0a754492f5cd35/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:14:58.452032 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-4bde84d0-5091-4a4f-abe6-317f2034cf8a None None] MFA Rules not processed for user `c788c31040454631a2f76aa1564a3650`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:58.452969 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dbdbe6-4d04-4ad9-b682-877ca69a7466 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:58.453323 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dbdbe6-4d04-4ad9-b682-877ca69a7466 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:58.453664 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dbdbe6-4d04-4ad9-b682-877ca69a7466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:58.453808 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-4bde84d0-5091-4a4f-abe6-317f2034cf8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:58.465450 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:58.516137 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-97dbdbe6-4d04-4ad9-b682-877ca69a7466 None None] MFA Rules not processed for user `0d56e72c69434537ad0a754492f5cd35`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:14:58.517465 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-97dbdbe6-4d04-4ad9-b682-877ca69a7466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:58.540699 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:14:58.557119 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 990/1979] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:58 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:58.567338 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bde84d0-5091-4a4f-abe6-317f2034cf8a None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:58.569240 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-4bde84d0-5091-4a4f-abe6-317f2034cf8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:58.631547 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 990/1980] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:14:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:14:58.649268 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dbdbe6-4d04-4ad9-b682-877ca69a7466 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:58.651478 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-97dbdbe6-4d04-4ad9-b682-877ca69a7466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:58.700596 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bde84d0-5091-4a4f-abe6-317f2034cf8a None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:58.705007 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3041b9f0-cb44-4bb7-8937-be592d3309be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:58.706664 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3041b9f0-cb44-4bb7-8937-be592d3309be service cinder] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:58.707151 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3041b9f0-cb44-4bb7-8937-be592d3309be service cinder] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:58.707822 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3041b9f0-cb44-4bb7-8937-be592d3309be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:58.721866 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3041b9f0-cb44-4bb7-8937-be592d3309be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:58.723861 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3041b9f0-cb44-4bb7-8937-be592d3309be service cinder] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:58.732277 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dbdbe6-4d04-4ad9-b682-877ca69a7466 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:58.736539 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472ccee3-e809-44df-8b97-af7afc3e374f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:58.737320 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ccee3-e809-44df-8b97-af7afc3e374f service cinder] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:58.737685 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 991/1981] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:14:58 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:14:58.738042 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ccee3-e809-44df-8b97-af7afc3e374f service cinder] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:58.738478 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ccee3-e809-44df-8b97-af7afc3e374f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:58.749701 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472ccee3-e809-44df-8b97-af7afc3e374f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:58.751393 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472ccee3-e809-44df-8b97-af7afc3e374f service cinder] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:58.772429 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 991/1982] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:14:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:14:58.929794 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3041b9f0-cb44-4bb7-8937-be592d3309be service cinder] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:14:58.950635 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3041b9f0-cb44-4bb7-8937-be592d3309be service cinder] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:14:58.969329 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ad0ef3-cbae-421c-be80-8362d24d79d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:14:58.970053 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad0ef3-cbae-421c-be80-8362d24d79d4 service cinder] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:14:58.970516 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad0ef3-cbae-421c-be80-8362d24d79d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:14:58.970940 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ad0ef3-cbae-421c-be80-8362d24d79d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:14:58.973626 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-93ad0ef3-cbae-421c-be80-8362d24d79d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:14:59.062323 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ad0ef3-cbae-421c-be80-8362d24d79d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:14:59.064018 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ad0ef3-cbae-421c-be80-8362d24d79d4 service cinder] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:14:59.086876 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 992/1983] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:14:58 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:15:00.806726 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472ccee3-e809-44df-8b97-af7afc3e374f service cinder] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:00.808193 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-472ccee3-e809-44df-8b97-af7afc3e374f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:00.891355 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472ccee3-e809-44df-8b97-af7afc3e374f service cinder] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:00.896134 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5898ee-bc5a-4fbe-99a1-ef38508860a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:00.896908 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5898ee-bc5a-4fbe-99a1-ef38508860a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:00.897415 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5898ee-bc5a-4fbe-99a1-ef38508860a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:00.898317 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5898ee-bc5a-4fbe-99a1-ef38508860a2 admin admin] PATH_INFO: `/v3/users/68f7291d033b428abf195b06e858fa3f` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:00.899888 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5898ee-bc5a-4fbe-99a1-ef38508860a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68f7291d033b428abf195b06e858fa3f)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:00.901949 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5898ee-bc5a-4fbe-99a1-ef38508860a2 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:00.953994 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-dc5898ee-bc5a-4fbe-99a1-ef38508860a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68f7291d033b428abf195b06e858fa3f'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:00.962772 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-dc5898ee-bc5a-4fbe-99a1-ef38508860a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68f7291d033b428abf195b06e858fa3f'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:00.965190 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:00.971748 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-dc5898ee-bc5a-4fbe-99a1-ef38508860a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68f7291d033b428abf195b06e858fa3f'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:00.972793 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-dc5898ee-bc5a-4fbe-99a1-ef38508860a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68f7291d033b428abf195b06e858fa3f'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:00.987656 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-dc5898ee-bc5a-4fbe-99a1-ef38508860a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68f7291d033b428abf195b06e858fa3f'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:00.994216 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 992/1984] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:15:00 2022] DELETE /identity/v3/users/68f7291d033b428abf195b06e858fa3f => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:00.998848 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c8693-8738-4eeb-b38e-85f2624ee7c9 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:00.999374 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c8693-8738-4eeb-b38e-85f2624ee7c9 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:00.999609 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c8693-8738-4eeb-b38e-85f2624ee7c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:01.019650 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-119c8693-8738-4eeb-b38e-85f2624ee7c9 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:01.020870 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-119c8693-8738-4eeb-b38e-85f2624ee7c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:01.068607 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:01.085300 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 993/1985] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:15:01.095276 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5898ee-bc5a-4fbe-99a1-ef38508860a2 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:01.096710 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-dc5898ee-bc5a-4fbe-99a1-ef38508860a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:01.153349 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5898ee-bc5a-4fbe-99a1-ef38508860a2 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:01.156968 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccfa345-91b4-4cf7-bb78-75fd2d8f1b9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:01.157572 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccfa345-91b4-4cf7-bb78-75fd2d8f1b9f service neutron] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:01.157855 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccfa345-91b4-4cf7-bb78-75fd2d8f1b9f service neutron] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:01.158130 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccfa345-91b4-4cf7-bb78-75fd2d8f1b9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:01.167719 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccfa345-91b4-4cf7-bb78-75fd2d8f1b9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:01.169143 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccfa345-91b4-4cf7-bb78-75fd2d8f1b9f service neutron] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:01.178533 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 993/1986] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:15:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:15:01.455778 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119c8693-8738-4eeb-b38e-85f2624ee7c9 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:01.458263 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-119c8693-8738-4eeb-b38e-85f2624ee7c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:01.471249 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119c8693-8738-4eeb-b38e-85f2624ee7c9 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:01.482571 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb1af1e-4083-4f00-926b-4683d27c99b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:01.483321 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb1af1e-4083-4f00-926b-4683d27c99b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:01.483608 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb1af1e-4083-4f00-926b-4683d27c99b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:01.483888 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb1af1e-4083-4f00-926b-4683d27c99b2 admin admin] PATH_INFO: `/v3/projects/a978fc319e6144888abbc0a1d6632cb2` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:01.485060 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb1af1e-4083-4f00-926b-4683d27c99b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a978fc319e6144888abbc0a1d6632cb2)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:01.486986 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb1af1e-4083-4f00-926b-4683d27c99b2 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:01.540925 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-9bb1af1e-4083-4f00-926b-4683d27c99b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a978fc319e6144888abbc0a1d6632cb2', 'request_id': 'req-9bb1af1e-4083-4f00-926b-4683d27c99b2'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:01.547551 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-9bb1af1e-4083-4f00-926b-4683d27c99b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a978fc319e6144888abbc0a1d6632cb2', 'request_id': 'req-9bb1af1e-4083-4f00-926b-4683d27c99b2'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:01.555911 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-9bb1af1e-4083-4f00-926b-4683d27c99b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a978fc319e6144888abbc0a1d6632cb2', 'request_id': 'req-9bb1af1e-4083-4f00-926b-4683d27c99b2'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:01.566667 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-9bb1af1e-4083-4f00-926b-4683d27c99b2 admin admin] The token cache is being invalidate because project a978fc319e6144888abbc0a1d6632cb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81712) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:15:01.566960 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-9bb1af1e-4083-4f00-926b-4683d27c99b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:01.567608 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-9bb1af1e-4083-4f00-926b-4683d27c99b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:01.569310 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 994/1987] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:15:01 2022] DELETE /identity/v3/projects/a978fc319e6144888abbc0a1d6632cb2 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:01.573456 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccfa345-91b4-4cf7-bb78-75fd2d8f1b9f service neutron] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:01.583976 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccfa345-91b4-4cf7-bb78-75fd2d8f1b9f service neutron] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:01.658352 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b13b05-0cd3-4e55-9628-26d299ef9982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:01.659148 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b13b05-0cd3-4e55-9628-26d299ef9982 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:01.659462 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b13b05-0cd3-4e55-9628-26d299ef9982 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:01.659738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b13b05-0cd3-4e55-9628-26d299ef9982 admin admin] PATH_INFO: `/v3/users/c76fe081eb6948ab9cdb636269f6abfd` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:01.660856 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b13b05-0cd3-4e55-9628-26d299ef9982 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c76fe081eb6948ab9cdb636269f6abfd)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:01.662875 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b13b05-0cd3-4e55-9628-26d299ef9982 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:01.711471 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-46b13b05-0cd3-4e55-9628-26d299ef9982 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c76fe081eb6948ab9cdb636269f6abfd'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:01.722226 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-46b13b05-0cd3-4e55-9628-26d299ef9982 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c76fe081eb6948ab9cdb636269f6abfd'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:01.730041 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-46b13b05-0cd3-4e55-9628-26d299ef9982 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c76fe081eb6948ab9cdb636269f6abfd'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:01.730715 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-46b13b05-0cd3-4e55-9628-26d299ef9982 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c76fe081eb6948ab9cdb636269f6abfd'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:01.745047 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-46b13b05-0cd3-4e55-9628-26d299ef9982 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c76fe081eb6948ab9cdb636269f6abfd'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:01.749011 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:01.752838 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 994/1988] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:15:01 2022] DELETE /identity/v3/users/c76fe081eb6948ab9cdb636269f6abfd => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:02.029090 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb1af1e-4083-4f00-926b-4683d27c99b2 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:02.030643 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-9bb1af1e-4083-4f00-926b-4683d27c99b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:02.053764 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb1af1e-4083-4f00-926b-4683d27c99b2 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:02.123518 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0a96ce-af3f-4519-9f8f-76c0a92ea7b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:02.124222 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a96ce-af3f-4519-9f8f-76c0a92ea7b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:02.124673 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a96ce-af3f-4519-9f8f-76c0a92ea7b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:02.124819 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a96ce-af3f-4519-9f8f-76c0a92ea7b1 admin admin] PATH_INFO: `/v3/projects/85c908b2bdc442b895a30c21201d6deb` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:02.126833 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0a96ce-af3f-4519-9f8f-76c0a92ea7b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85c908b2bdc442b895a30c21201d6deb)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:02.128873 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0a96ce-af3f-4519-9f8f-76c0a92ea7b1 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:02.184976 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-0e0a96ce-af3f-4519-9f8f-76c0a92ea7b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85c908b2bdc442b895a30c21201d6deb', 'request_id': 'req-0e0a96ce-af3f-4519-9f8f-76c0a92ea7b1'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:02.195851 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-0e0a96ce-af3f-4519-9f8f-76c0a92ea7b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85c908b2bdc442b895a30c21201d6deb', 'request_id': 'req-0e0a96ce-af3f-4519-9f8f-76c0a92ea7b1'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:02.208190 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-0e0a96ce-af3f-4519-9f8f-76c0a92ea7b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85c908b2bdc442b895a30c21201d6deb', 'request_id': 'req-0e0a96ce-af3f-4519-9f8f-76c0a92ea7b1'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:02.232274 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-0e0a96ce-af3f-4519-9f8f-76c0a92ea7b1 admin admin] The token cache is being invalidate because project 85c908b2bdc442b895a30c21201d6deb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81712) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:15:02.233124 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-0e0a96ce-af3f-4519-9f8f-76c0a92ea7b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:02.234148 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-0e0a96ce-af3f-4519-9f8f-76c0a92ea7b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:02.237569 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 995/1989] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:15:02 2022] DELETE /identity/v3/projects/85c908b2bdc442b895a30c21201d6deb => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:02.244274 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b13b05-0cd3-4e55-9628-26d299ef9982 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:02.246764 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-46b13b05-0cd3-4e55-9628-26d299ef9982 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:02.259980 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b13b05-0cd3-4e55-9628-26d299ef9982 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:02.322018 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa36021d-5223-4723-aef2-23bfbb5a9b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:02.322577 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa36021d-5223-4723-aef2-23bfbb5a9b2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:02.322855 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa36021d-5223-4723-aef2-23bfbb5a9b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:02.323122 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa36021d-5223-4723-aef2-23bfbb5a9b2d admin admin] PATH_INFO: `/v3/users/1cd7b58d61ed404694f73235a3d8c286` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:02.324266 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa36021d-5223-4723-aef2-23bfbb5a9b2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cd7b58d61ed404694f73235a3d8c286)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:02.326159 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa36021d-5223-4723-aef2-23bfbb5a9b2d admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:02.371922 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-fa36021d-5223-4723-aef2-23bfbb5a9b2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cd7b58d61ed404694f73235a3d8c286'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:02.379142 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-fa36021d-5223-4723-aef2-23bfbb5a9b2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cd7b58d61ed404694f73235a3d8c286'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:02.386681 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-fa36021d-5223-4723-aef2-23bfbb5a9b2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cd7b58d61ed404694f73235a3d8c286'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:02.387335 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-fa36021d-5223-4723-aef2-23bfbb5a9b2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cd7b58d61ed404694f73235a3d8c286'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:02.398942 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-fa36021d-5223-4723-aef2-23bfbb5a9b2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cd7b58d61ed404694f73235a3d8c286'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:02.404647 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 995/1990] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:15:02 2022] DELETE /identity/v3/users/1cd7b58d61ed404694f73235a3d8c286 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:02.673202 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0a96ce-af3f-4519-9f8f-76c0a92ea7b1 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:02.674684 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-0e0a96ce-af3f-4519-9f8f-76c0a92ea7b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:02.683468 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:02.689641 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0a96ce-af3f-4519-9f8f-76c0a92ea7b1 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:02.760199 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3029b3cc-4bd1-4c1f-8cb0-fe64f555251c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:02.760842 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3029b3cc-4bd1-4c1f-8cb0-fe64f555251c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:02.761130 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3029b3cc-4bd1-4c1f-8cb0-fe64f555251c admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:02.761406 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3029b3cc-4bd1-4c1f-8cb0-fe64f555251c admin admin] PATH_INFO: `/v3/projects/7f7f322acdca49e28fc25e70044dbcf6` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:02.762939 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3029b3cc-4bd1-4c1f-8cb0-fe64f555251c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f7f322acdca49e28fc25e70044dbcf6)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:02.764931 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3029b3cc-4bd1-4c1f-8cb0-fe64f555251c admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:02.812524 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-3029b3cc-4bd1-4c1f-8cb0-fe64f555251c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f7f322acdca49e28fc25e70044dbcf6', 'request_id': 'req-3029b3cc-4bd1-4c1f-8cb0-fe64f555251c'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:02.819047 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-3029b3cc-4bd1-4c1f-8cb0-fe64f555251c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f7f322acdca49e28fc25e70044dbcf6', 'request_id': 'req-3029b3cc-4bd1-4c1f-8cb0-fe64f555251c'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:02.827914 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-3029b3cc-4bd1-4c1f-8cb0-fe64f555251c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f7f322acdca49e28fc25e70044dbcf6', 'request_id': 'req-3029b3cc-4bd1-4c1f-8cb0-fe64f555251c'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:02.841039 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-3029b3cc-4bd1-4c1f-8cb0-fe64f555251c admin admin] The token cache is being invalidate because project 7f7f322acdca49e28fc25e70044dbcf6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81712) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:15:02.841335 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-3029b3cc-4bd1-4c1f-8cb0-fe64f555251c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:02.841985 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-3029b3cc-4bd1-4c1f-8cb0-fe64f555251c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:02.846967 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 996/1991] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:15:02 2022] DELETE /identity/v3/projects/7f7f322acdca49e28fc25e70044dbcf6 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:02.848637 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5293c971-9ac7-4299-8d9a-e10549b59629 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:02.848926 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5293c971-9ac7-4299-8d9a-e10549b59629 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:02.849206 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5293c971-9ac7-4299-8d9a-e10549b59629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:02.884970 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-5293c971-9ac7-4299-8d9a-e10549b59629 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:02.886201 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-5293c971-9ac7-4299-8d9a-e10549b59629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:02.949370 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 996/1992] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:15:02.958697 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5120c439-7d57-4d3d-b067-fa427a023917 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:02.958978 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5120c439-7d57-4d3d-b067-fa427a023917 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:02.959309 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5120c439-7d57-4d3d-b067-fa427a023917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:02.980149 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-5120c439-7d57-4d3d-b067-fa427a023917 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:02.981418 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-5120c439-7d57-4d3d-b067-fa427a023917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:02.993543 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 997/1993] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:15:02.994977 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5293c971-9ac7-4299-8d9a-e10549b59629 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:03.005085 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5293c971-9ac7-4299-8d9a-e10549b59629 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:03.010774 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9a19fe-a054-4bf4-b305-c9c108d24f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:03.011481 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a19fe-a054-4bf4-b305-c9c108d24f38 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:03.011746 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a19fe-a054-4bf4-b305-c9c108d24f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:03.012023 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a19fe-a054-4bf4-b305-c9c108d24f38 admin admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:03.012705 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9a19fe-a054-4bf4-b305-c9c108d24f38 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:15:03.013008 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9a19fe-a054-4bf4-b305-c9c108d24f38 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:03.014600 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9a19fe-a054-4bf4-b305-c9c108d24f38 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:03.028968 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 997/1994] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:15:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:15:03.033388 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5120c439-7d57-4d3d-b067-fa427a023917 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:03.043569 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5120c439-7d57-4d3d-b067-fa427a023917 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:03.049361 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32751a1e-9404-4b3a-827f-e303c67482d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:03.050043 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32751a1e-9404-4b3a-827f-e303c67482d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:03.050397 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32751a1e-9404-4b3a-827f-e303c67482d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:03.050738 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32751a1e-9404-4b3a-827f-e303c67482d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:03.051474 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32751a1e-9404-4b3a-827f-e303c67482d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:03.053437 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32751a1e-9404-4b3a-827f-e303c67482d8 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:03.083087 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 998/1995] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:15:03 2022] POST /identity/v3/projects => generated 365 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:15:03.087220 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9a19fe-a054-4bf4-b305-c9c108d24f38 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:03.092364 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:03.098323 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9a19fe-a054-4bf4-b305-c9c108d24f38 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:03.158900 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4206e37b-a3f1-41dd-99ab-962e97445b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:03.159551 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4206e37b-a3f1-41dd-99ab-962e97445b22 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:03.159833 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4206e37b-a3f1-41dd-99ab-962e97445b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:03.160105 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4206e37b-a3f1-41dd-99ab-962e97445b22 admin admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:03.160739 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4206e37b-a3f1-41dd-99ab-962e97445b22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:03.162710 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4206e37b-a3f1-41dd-99ab-962e97445b22 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:03.184286 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 998/1996] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:15:03 2022] POST /identity/v3/users => generated 283 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 12:15:03.188798 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32751a1e-9404-4b3a-827f-e303c67482d8 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:03.200620 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32751a1e-9404-4b3a-827f-e303c67482d8 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:03.207674 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f8d0cf-73f2-41cb-b2d1-e5496c474252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:03.208351 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f8d0cf-73f2-41cb-b2d1-e5496c474252 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:03.208717 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f8d0cf-73f2-41cb-b2d1-e5496c474252 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:03.208877 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f8d0cf-73f2-41cb-b2d1-e5496c474252 admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:03.209619 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f8d0cf-73f2-41cb-b2d1-e5496c474252 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:03.211474 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f8d0cf-73f2-41cb-b2d1-e5496c474252 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:03.223508 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 999/1997] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:15:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:15:03.227716 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4206e37b-a3f1-41dd-99ab-962e97445b22 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:03.240004 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4206e37b-a3f1-41dd-99ab-962e97445b22 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:03.246008 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6057caf-48ca-4981-9e13-da0856a61ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:03.247340 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6057caf-48ca-4981-9e13-da0856a61ded admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:03.247631 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6057caf-48ca-4981-9e13-da0856a61ded admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:03.247905 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6057caf-48ca-4981-9e13-da0856a61ded admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:03.248913 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6057caf-48ca-4981-9e13-da0856a61ded admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:03.250813 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6057caf-48ca-4981-9e13-da0856a61ded admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:03.264932 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 999/1998] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:15:03 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 12:15:03.269499 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f8d0cf-73f2-41cb-b2d1-e5496c474252 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:03.281253 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f8d0cf-73f2-41cb-b2d1-e5496c474252 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:03.288333 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0dd687-0250-4a49-b4a9-0809fda7cd5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:03.288333 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0dd687-0250-4a49-b4a9-0809fda7cd5c admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:03.288333 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0dd687-0250-4a49-b4a9-0809fda7cd5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:03.289128 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0dd687-0250-4a49-b4a9-0809fda7cd5c admin admin] PATH_INFO: `/v3/projects/65d237e0efb8470caf0b20790088afda/users/53c54f06cbb54dd48e3d99fae89fc107/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:03.328076 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0dd687-0250-4a49-b4a9-0809fda7cd5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=65d237e0efb8470caf0b20790088afda, user_id=53c54f06cbb54dd48e3d99fae89fc107, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:03.333073 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0dd687-0250-4a49-b4a9-0809fda7cd5c admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:03.376637 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1000/1999] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:15:03 2022] PUT /identity/v3/projects/65d237e0efb8470caf0b20790088afda/users/53c54f06cbb54dd48e3d99fae89fc107/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:03.387725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df59588-3a0c-449a-95d6-9d387a3c4f17 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:03.388105 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df59588-3a0c-449a-95d6-9d387a3c4f17 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:03.388690 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df59588-3a0c-449a-95d6-9d387a3c4f17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:03.451098 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-3df59588-3a0c-449a-95d6-9d387a3c4f17 None None] MFA Rules not processed for user `53c54f06cbb54dd48e3d99fae89fc107`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:03.454462 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-3df59588-3a0c-449a-95d6-9d387a3c4f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:03.573266 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1000/2000] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:03 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:15:03.584730 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0dd687-0250-4a49-b4a9-0809fda7cd5c admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:03.587971 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-5d0dd687-0250-4a49-b4a9-0809fda7cd5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:03.637290 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:03.665392 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0dd687-0250-4a49-b4a9-0809fda7cd5c admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:03.669117 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7755e65d-f57a-4b47-bac4-473d56dfd824 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:03.669828 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7755e65d-f57a-4b47-bac4-473d56dfd824 service cinder] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:03.670116 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7755e65d-f57a-4b47-bac4-473d56dfd824 service cinder] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:03.670381 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7755e65d-f57a-4b47-bac4-473d56dfd824 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:03.684524 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7755e65d-f57a-4b47-bac4-473d56dfd824 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:03.685977 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7755e65d-f57a-4b47-bac4-473d56dfd824 service cinder] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:03.703325 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1001/2001] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:15:03 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 12:15:03.931860 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df59588-3a0c-449a-95d6-9d387a3c4f17 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:03.933304 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-3df59588-3a0c-449a-95d6-9d387a3c4f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:04.035607 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df59588-3a0c-449a-95d6-9d387a3c4f17 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:04.041136 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f6f7bd-9b60-4fc5-aa1d-5d9a716fabc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:04.041919 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f6f7bd-9b60-4fc5-aa1d-5d9a716fabc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:04.042254 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f6f7bd-9b60-4fc5-aa1d-5d9a716fabc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:04.042719 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f6f7bd-9b60-4fc5-aa1d-5d9a716fabc8 admin admin] PATH_INFO: `/v3/users/b0f5701503774a03ab9ca24982c18d35` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:04.044055 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f6f7bd-9b60-4fc5-aa1d-5d9a716fabc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0f5701503774a03ab9ca24982c18d35)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:04.046132 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f6f7bd-9b60-4fc5-aa1d-5d9a716fabc8 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:04.097990 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:04.121227 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7755e65d-f57a-4b47-bac4-473d56dfd824 service cinder] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:04.122782 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-7755e65d-f57a-4b47-bac4-473d56dfd824 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:04.142402 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7755e65d-f57a-4b47-bac4-473d56dfd824 service cinder] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:04.148549 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b6d78-af7c-472f-83f8-278b7678cbae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:04.149389 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b6d78-af7c-472f-83f8-278b7678cbae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:04.149674 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b6d78-af7c-472f-83f8-278b7678cbae admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:04.149953 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b6d78-af7c-472f-83f8-278b7678cbae admin admin] PATH_INFO: `/v3/users/0d56e72c69434537ad0a754492f5cd35` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:04.151326 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4b6d78-af7c-472f-83f8-278b7678cbae admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d56e72c69434537ad0a754492f5cd35)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:04.153488 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4b6d78-af7c-472f-83f8-278b7678cbae admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:04.168579 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-10f6f7bd-9b60-4fc5-aa1d-5d9a716fabc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0f5701503774a03ab9ca24982c18d35'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:04.190397 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-10f6f7bd-9b60-4fc5-aa1d-5d9a716fabc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0f5701503774a03ab9ca24982c18d35'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:04.200518 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-10f6f7bd-9b60-4fc5-aa1d-5d9a716fabc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0f5701503774a03ab9ca24982c18d35'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:04.201318 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-10f6f7bd-9b60-4fc5-aa1d-5d9a716fabc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0f5701503774a03ab9ca24982c18d35'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:04.217393 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-10f6f7bd-9b60-4fc5-aa1d-5d9a716fabc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0f5701503774a03ab9ca24982c18d35'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:04.218403 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-7b4b6d78-af7c-472f-83f8-278b7678cbae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d56e72c69434537ad0a754492f5cd35'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:04.224164 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1001/2002] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:15:03 2022] DELETE /identity/v3/users/b0f5701503774a03ab9ca24982c18d35 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:04.226786 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-7b4b6d78-af7c-472f-83f8-278b7678cbae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d56e72c69434537ad0a754492f5cd35'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:04.227492 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-7b4b6d78-af7c-472f-83f8-278b7678cbae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d56e72c69434537ad0a754492f5cd35'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:04.229645 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f1f683-aedc-4db5-9808-ee5849fa8cf9 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:04.230177 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f1f683-aedc-4db5-9808-ee5849fa8cf9 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:04.230177 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f1f683-aedc-4db5-9808-ee5849fa8cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:04.236503 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-7b4b6d78-af7c-472f-83f8-278b7678cbae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d56e72c69434537ad0a754492f5cd35'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:04.237182 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-7b4b6d78-af7c-472f-83f8-278b7678cbae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d56e72c69434537ad0a754492f5cd35'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:04.252546 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-a6f1f683-aedc-4db5-9808-ee5849fa8cf9 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:04.254595 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-a6f1f683-aedc-4db5-9808-ee5849fa8cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:04.262344 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1002/2003] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:15:04 2022] DELETE /identity/v3/users/0d56e72c69434537ad0a754492f5cd35 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:04.268100 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4808289b-725b-4584-95cf-2d2f67267098 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:04.268390 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4808289b-725b-4584-95cf-2d2f67267098 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:04.268669 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4808289b-725b-4584-95cf-2d2f67267098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:04.290139 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-4808289b-725b-4584-95cf-2d2f67267098 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:04.291407 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-4808289b-725b-4584-95cf-2d2f67267098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:04.337107 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1002/2004] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:15:04.347187 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f1f683-aedc-4db5-9808-ee5849fa8cf9 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:04.349025 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-a6f1f683-aedc-4db5-9808-ee5849fa8cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:04.362629 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1003/2005] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:15:04.375850 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4808289b-725b-4584-95cf-2d2f67267098 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:04.378496 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-4808289b-725b-4584-95cf-2d2f67267098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:04.412043 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f1f683-aedc-4db5-9808-ee5849fa8cf9 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:04.439320 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:04.453961 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4808289b-725b-4584-95cf-2d2f67267098 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:04.459144 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1db354-15d1-4df3-a935-de2935fd64e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:04.459790 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1db354-15d1-4df3-a935-de2935fd64e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:04.461076 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1db354-15d1-4df3-a935-de2935fd64e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:04.461595 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1db354-15d1-4df3-a935-de2935fd64e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:04.465957 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:04.472547 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1db354-15d1-4df3-a935-de2935fd64e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:04.474086 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1db354-15d1-4df3-a935-de2935fd64e0 service neutron] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:04.475477 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adc7170-40dc-4f1d-9c66-a61a032d2258 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:04.476430 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adc7170-40dc-4f1d-9c66-a61a032d2258 service neutron] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:04.477128 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adc7170-40dc-4f1d-9c66-a61a032d2258 service neutron] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:04.477876 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adc7170-40dc-4f1d-9c66-a61a032d2258 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:04.485655 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1004/2006] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:15:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 12:15:04.490325 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adc7170-40dc-4f1d-9c66-a61a032d2258 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:04.492723 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adc7170-40dc-4f1d-9c66-a61a032d2258 service neutron] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:04.503692 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1003/2007] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:15:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 12:15:04.998675 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1db354-15d1-4df3-a935-de2935fd64e0 service neutron] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:05.001377 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-df1db354-15d1-4df3-a935-de2935fd64e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:05.019539 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1db354-15d1-4df3-a935-de2935fd64e0 service neutron] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:05.027722 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6188961-489c-421e-8004-ea8abfd764ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:05.029020 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6188961-489c-421e-8004-ea8abfd764ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:05.029549 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6188961-489c-421e-8004-ea8abfd764ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:05.030171 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6188961-489c-421e-8004-ea8abfd764ed admin admin] PATH_INFO: `/v3/projects/afc067f6c1e741d5a04b278d67dba2a6` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:05.032762 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6188961-489c-421e-8004-ea8abfd764ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=afc067f6c1e741d5a04b278d67dba2a6)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:05.036972 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6188961-489c-421e-8004-ea8abfd764ed admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:05.057217 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adc7170-40dc-4f1d-9c66-a61a032d2258 service neutron] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:05.058726 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-5adc7170-40dc-4f1d-9c66-a61a032d2258 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:05.080453 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adc7170-40dc-4f1d-9c66-a61a032d2258 service neutron] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:05.095695 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8ca35d-a775-40a6-b802-a913c249aa28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:05.096251 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ca35d-a775-40a6-b802-a913c249aa28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:05.096528 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ca35d-a775-40a6-b802-a913c249aa28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:05.096800 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ca35d-a775-40a6-b802-a913c249aa28 admin admin] PATH_INFO: `/v3/projects/ceb6d267cb364989a4767c22eed4defd` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:05.101289 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8ca35d-a775-40a6-b802-a913c249aa28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ceb6d267cb364989a4767c22eed4defd)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:05.105175 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8ca35d-a775-40a6-b802-a913c249aa28 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:05.111243 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-a6188961-489c-421e-8004-ea8abfd764ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afc067f6c1e741d5a04b278d67dba2a6', 'request_id': 'req-a6188961-489c-421e-8004-ea8abfd764ed'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:05.120170 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-a6188961-489c-421e-8004-ea8abfd764ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afc067f6c1e741d5a04b278d67dba2a6', 'request_id': 'req-a6188961-489c-421e-8004-ea8abfd764ed'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:05.130806 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-a6188961-489c-421e-8004-ea8abfd764ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afc067f6c1e741d5a04b278d67dba2a6', 'request_id': 'req-a6188961-489c-421e-8004-ea8abfd764ed'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:05.146024 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-a6188961-489c-421e-8004-ea8abfd764ed admin admin] The token cache is being invalidate because project afc067f6c1e741d5a04b278d67dba2a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81712) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:15:05.146308 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-a6188961-489c-421e-8004-ea8abfd764ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:05.146960 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-a6188961-489c-421e-8004-ea8abfd764ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:05.149049 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1005/2008] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:15:04 2022] DELETE /identity/v3/projects/afc067f6c1e741d5a04b278d67dba2a6 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:05.153924 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346482fc-9df0-4a5e-967d-100a91fa1059 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:05.154424 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346482fc-9df0-4a5e-967d-100a91fa1059 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:05.154875 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346482fc-9df0-4a5e-967d-100a91fa1059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:05.186160 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-346482fc-9df0-4a5e-967d-100a91fa1059 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:05.187789 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-346482fc-9df0-4a5e-967d-100a91fa1059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:05.212487 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-7f8ca35d-a775-40a6-b802-a913c249aa28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ceb6d267cb364989a4767c22eed4defd', 'request_id': 'req-7f8ca35d-a775-40a6-b802-a913c249aa28'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:05.226858 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-7f8ca35d-a775-40a6-b802-a913c249aa28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ceb6d267cb364989a4767c22eed4defd', 'request_id': 'req-7f8ca35d-a775-40a6-b802-a913c249aa28'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:05.240883 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-7f8ca35d-a775-40a6-b802-a913c249aa28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ceb6d267cb364989a4767c22eed4defd', 'request_id': 'req-7f8ca35d-a775-40a6-b802-a913c249aa28'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:05.267371 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-7f8ca35d-a775-40a6-b802-a913c249aa28 admin admin] The token cache is being invalidate because project ceb6d267cb364989a4767c22eed4defd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:15:05.268005 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-7f8ca35d-a775-40a6-b802-a913c249aa28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:05.268781 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-7f8ca35d-a775-40a6-b802-a913c249aa28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:05.276689 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1004/2009] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:15:05 2022] DELETE /identity/v3/projects/ceb6d267cb364989a4767c22eed4defd => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 12:15:05.277811 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8ca35d-a775-40a6-b802-a913c249aa28 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:05.279411 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-7f8ca35d-a775-40a6-b802-a913c249aa28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:05.321939 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1006/2010] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:15:05.331274 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2952a18e-0601-4b60-a6b3-c634089ef6de None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:05.331817 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2952a18e-0601-4b60-a6b3-c634089ef6de None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:05.332107 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2952a18e-0601-4b60-a6b3-c634089ef6de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:05.395679 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-2952a18e-0601-4b60-a6b3-c634089ef6de None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:05.398332 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-2952a18e-0601-4b60-a6b3-c634089ef6de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:05.412089 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1007/2011] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:15:05.417542 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2952a18e-0601-4b60-a6b3-c634089ef6de None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:05.422899 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8ca35d-a775-40a6-b802-a913c249aa28 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:05.438230 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9e6eeb-c2b2-4622-b2e6-7ed0d898ab4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:05.439059 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e6eeb-c2b2-4622-b2e6-7ed0d898ab4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:05.439943 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e6eeb-c2b2-4622-b2e6-7ed0d898ab4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:05.439943 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e6eeb-c2b2-4622-b2e6-7ed0d898ab4e admin admin] PATH_INFO: `/v3/users/c788c31040454631a2f76aa1564a3650` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:05.441843 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9e6eeb-c2b2-4622-b2e6-7ed0d898ab4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c788c31040454631a2f76aa1564a3650)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:05.444179 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9e6eeb-c2b2-4622-b2e6-7ed0d898ab4e admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:05.463808 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2952a18e-0601-4b60-a6b3-c634089ef6de None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:05.493532 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7745d512-62a1-4c31-966c-e23a61bd5dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:05.494166 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7745d512-62a1-4c31-966c-e23a61bd5dc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:05.494838 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7745d512-62a1-4c31-966c-e23a61bd5dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:05.494838 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7745d512-62a1-4c31-966c-e23a61bd5dc5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:05.496248 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7745d512-62a1-4c31-966c-e23a61bd5dc5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:15:05.496553 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7745d512-62a1-4c31-966c-e23a61bd5dc5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:05.498282 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7745d512-62a1-4c31-966c-e23a61bd5dc5 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:05.532240 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-3c9e6eeb-c2b2-4622-b2e6-7ed0d898ab4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c788c31040454631a2f76aa1564a3650'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:05.544266 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1008/2012] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:15:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:15:05.549768 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-3c9e6eeb-c2b2-4622-b2e6-7ed0d898ab4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c788c31040454631a2f76aa1564a3650'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:05.551464 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7745d512-62a1-4c31-966c-e23a61bd5dc5 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:05.568825 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-3c9e6eeb-c2b2-4622-b2e6-7ed0d898ab4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c788c31040454631a2f76aa1564a3650'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:05.572016 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-3c9e6eeb-c2b2-4622-b2e6-7ed0d898ab4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c788c31040454631a2f76aa1564a3650'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:05.572685 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7745d512-62a1-4c31-966c-e23a61bd5dc5 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:05.578936 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c5b7e5-8dcf-4774-a590-2f5571e29f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:05.579948 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c5b7e5-8dcf-4774-a590-2f5571e29f60 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:05.580566 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c5b7e5-8dcf-4774-a590-2f5571e29f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:05.581632 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c5b7e5-8dcf-4774-a590-2f5571e29f60 admin admin] PATH_INFO: `/v3/projects` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:05.582105 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:05.583076 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c5b7e5-8dcf-4774-a590-2f5571e29f60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:05.585620 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c5b7e5-8dcf-4774-a590-2f5571e29f60 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:05.598205 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-3c9e6eeb-c2b2-4622-b2e6-7ed0d898ab4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c788c31040454631a2f76aa1564a3650'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:05.621286 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1005/2013] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:15:05 2022] DELETE /identity/v3/users/c788c31040454631a2f76aa1564a3650 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:05.642433 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1009/2014] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:15:05 2022] POST /identity/v3/projects => generated 383 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:15:05.649387 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9e6eeb-c2b2-4622-b2e6-7ed0d898ab4e admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:05.652091 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-3c9e6eeb-c2b2-4622-b2e6-7ed0d898ab4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:05.772159 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9e6eeb-c2b2-4622-b2e6-7ed0d898ab4e admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:05.872726 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca03362-e096-428b-aad0-7bb429e615df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:05.873348 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca03362-e096-428b-aad0-7bb429e615df admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:05.874231 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca03362-e096-428b-aad0-7bb429e615df admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:05.874520 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca03362-e096-428b-aad0-7bb429e615df admin admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:05.875161 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca03362-e096-428b-aad0-7bb429e615df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:05.877556 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca03362-e096-428b-aad0-7bb429e615df admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:05.909874 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1006/2015] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:15:05 2022] POST /identity/v3/users => generated 292 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:15:05.916342 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c5b7e5-8dcf-4774-a590-2f5571e29f60 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:05.934282 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c5b7e5-8dcf-4774-a590-2f5571e29f60 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:05.942197 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7cfa17-6450-47f5-8fc1-2f1266c61bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:05.943180 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7cfa17-6450-47f5-8fc1-2f1266c61bce admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:05.943498 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7cfa17-6450-47f5-8fc1-2f1266c61bce admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:05.943975 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7cfa17-6450-47f5-8fc1-2f1266c61bce admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:05.944842 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7cfa17-6450-47f5-8fc1-2f1266c61bce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:05.946868 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7cfa17-6450-47f5-8fc1-2f1266c61bce admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:05.969827 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1010/2016] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:15:05 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 12:15:05.971391 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca03362-e096-428b-aad0-7bb429e615df admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:05.993830 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca03362-e096-428b-aad0-7bb429e615df admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:06.000382 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca8d3ca-679f-4f16-998c-608543474136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:06.001146 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8d3ca-679f-4f16-998c-608543474136 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:06.001491 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8d3ca-679f-4f16-998c-608543474136 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:06.001881 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8d3ca-679f-4f16-998c-608543474136 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:06.002748 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca8d3ca-679f-4f16-998c-608543474136 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:06.004570 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca8d3ca-679f-4f16-998c-608543474136 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:06.026971 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1007/2017] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:15:05 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:15:06.034232 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7cfa17-6450-47f5-8fc1-2f1266c61bce admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:06.056282 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca8d3ca-679f-4f16-998c-608543474136 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:06.058041 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-7ca8d3ca-679f-4f16-998c-608543474136 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:06.069003 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7cfa17-6450-47f5-8fc1-2f1266c61bce admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:06.094838 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29819de-ad68-48b6-886e-d09cabfc2c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:06.100565 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca8d3ca-679f-4f16-998c-608543474136 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:06.101735 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29819de-ad68-48b6-886e-d09cabfc2c2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:06.102016 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29819de-ad68-48b6-886e-d09cabfc2c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:06.102291 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29819de-ad68-48b6-886e-d09cabfc2c2b admin admin] PATH_INFO: `/v3/projects/59e2b4755fa64f61910a6e4fe5cfb8ed/users/f2a8f617fed54d649d058fa43aa50314/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:06.123088 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf6e734-c2f2-4c2a-87a3-f0b2d4030118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:06.124057 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6e734-c2f2-4c2a-87a3-f0b2d4030118 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:06.124642 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6e734-c2f2-4c2a-87a3-f0b2d4030118 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:06.125638 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6e734-c2f2-4c2a-87a3-f0b2d4030118 admin admin] PATH_INFO: `/v3/projects/ab07e470aded4662afbf69833ce7932d` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:06.128686 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf6e734-c2f2-4c2a-87a3-f0b2d4030118 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab07e470aded4662afbf69833ce7932d)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:06.131553 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf6e734-c2f2-4c2a-87a3-f0b2d4030118 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:06.157898 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29819de-ad68-48b6-886e-d09cabfc2c2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=59e2b4755fa64f61910a6e4fe5cfb8ed, user_id=f2a8f617fed54d649d058fa43aa50314, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:06.168393 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29819de-ad68-48b6-886e-d09cabfc2c2b admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:06.181612 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:06.208857 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1011/2018] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:15:06 2022] PUT /identity/v3/projects/59e2b4755fa64f61910a6e4fe5cfb8ed/users/f2a8f617fed54d649d058fa43aa50314/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:06.213614 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5929a884-463d-4c6a-b9d1-1d87b36aa440 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:06.214205 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5929a884-463d-4c6a-b9d1-1d87b36aa440 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:06.214604 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5929a884-463d-4c6a-b9d1-1d87b36aa440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:06.266234 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-6bf6e734-c2f2-4c2a-87a3-f0b2d4030118 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab07e470aded4662afbf69833ce7932d', 'request_id': 'req-6bf6e734-c2f2-4c2a-87a3-f0b2d4030118'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:06.279764 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-6bf6e734-c2f2-4c2a-87a3-f0b2d4030118 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab07e470aded4662afbf69833ce7932d', 'request_id': 'req-6bf6e734-c2f2-4c2a-87a3-f0b2d4030118'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:06.291621 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-5929a884-463d-4c6a-b9d1-1d87b36aa440 None None] MFA Rules not processed for user `f2a8f617fed54d649d058fa43aa50314`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:06.293174 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-5929a884-463d-4c6a-b9d1-1d87b36aa440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:06.296641 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-6bf6e734-c2f2-4c2a-87a3-f0b2d4030118 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab07e470aded4662afbf69833ce7932d', 'request_id': 'req-6bf6e734-c2f2-4c2a-87a3-f0b2d4030118'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:06.308690 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=88,keystone:DELETE=8,keystone:INSERT=4 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:06.330687 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-6bf6e734-c2f2-4c2a-87a3-f0b2d4030118 admin admin] The token cache is being invalidate because project ab07e470aded4662afbf69833ce7932d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:15:06.330943 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-6bf6e734-c2f2-4c2a-87a3-f0b2d4030118 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:06.333199 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-6bf6e734-c2f2-4c2a-87a3-f0b2d4030118 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:06.340587 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1008/2019] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:15:06 2022] DELETE /identity/v3/projects/ab07e470aded4662afbf69833ce7932d => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:06.346921 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c290f8c2-350c-42e3-bbf9-4edea3e23b78 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:06.347158 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c290f8c2-350c-42e3-bbf9-4edea3e23b78 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:06.347473 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c290f8c2-350c-42e3-bbf9-4edea3e23b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:06.399791 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-c290f8c2-350c-42e3-bbf9-4edea3e23b78 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:06.401040 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-c290f8c2-350c-42e3-bbf9-4edea3e23b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:06.431318 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1012/2020] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:15:06.438635 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5929a884-463d-4c6a-b9d1-1d87b36aa440 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:06.442468 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-5929a884-463d-4c6a-b9d1-1d87b36aa440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:06.550786 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1009/2021] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:15:06.566418 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4533f7ed-38b2-40c3-9077-1f2875c9b175 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:06.566951 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4533f7ed-38b2-40c3-9077-1f2875c9b175 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:06.570293 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4533f7ed-38b2-40c3-9077-1f2875c9b175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:06.589829 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5929a884-463d-4c6a-b9d1-1d87b36aa440 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:06.600711 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba85b06-c78c-4d9f-8585-f75b05355d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:06.602395 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba85b06-c78c-4d9f-8585-f75b05355d2d admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:06.602688 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba85b06-c78c-4d9f-8585-f75b05355d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:06.602966 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba85b06-c78c-4d9f-8585-f75b05355d2d admin admin] PATH_INFO: `/v3/projects` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:06.604720 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba85b06-c78c-4d9f-8585-f75b05355d2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:06.608911 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba85b06-c78c-4d9f-8585-f75b05355d2d admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:06.621556 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-4533f7ed-38b2-40c3-9077-1f2875c9b175 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:06.623720 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-4533f7ed-38b2-40c3-9077-1f2875c9b175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:06.636577 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1010/2022] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:15:06.642737 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4533f7ed-38b2-40c3-9077-1f2875c9b175 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:06.650589 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1013/2023] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:15:06 2022] POST /identity/v3/projects => generated 385 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:15:06.656459 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba85b06-c78c-4d9f-8585-f75b05355d2d admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:06.662658 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4533f7ed-38b2-40c3-9077-1f2875c9b175 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:06.799960 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba85b06-c78c-4d9f-8585-f75b05355d2d admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:06.814474 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6780ca5e-96f6-4df7-ad24-bc2b1a0681df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:06.815317 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6780ca5e-96f6-4df7-ad24-bc2b1a0681df admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:06.815677 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6780ca5e-96f6-4df7-ad24-bc2b1a0681df admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:06.815864 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6780ca5e-96f6-4df7-ad24-bc2b1a0681df admin admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:06.816511 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6780ca5e-96f6-4df7-ad24-bc2b1a0681df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:06.819310 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6780ca5e-96f6-4df7-ad24-bc2b1a0681df admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:06.821486 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b276ef2e-3864-466c-ae81-b99f15587be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:06.822438 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b276ef2e-3864-466c-ae81-b99f15587be7 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:06.822614 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b276ef2e-3864-466c-ae81-b99f15587be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:06.823148 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b276ef2e-3864-466c-ae81-b99f15587be7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:06.823584 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b276ef2e-3864-466c-ae81-b99f15587be7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:15:06.824279 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b276ef2e-3864-466c-ae81-b99f15587be7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:06.826464 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b276ef2e-3864-466c-ae81-b99f15587be7 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:06.856151 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1011/2024] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:15:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:15:06.857252 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b276ef2e-3864-466c-ae81-b99f15587be7 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:06.866793 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:06.880072 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1014/2025] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:15:06 2022] POST /identity/v3/users => generated 293 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:15:06.884533 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6780ca5e-96f6-4df7-ad24-bc2b1a0681df admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:06.886958 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b276ef2e-3864-466c-ae81-b99f15587be7 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:06.901317 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6780ca5e-96f6-4df7-ad24-bc2b1a0681df admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:06.906274 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81326186-4821-42e0-a10b-a4b72b32d9f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:06.906895 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81326186-4821-42e0-a10b-a4b72b32d9f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:06.907177 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81326186-4821-42e0-a10b-a4b72b32d9f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:06.907484 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81326186-4821-42e0-a10b-a4b72b32d9f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:06.908136 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81326186-4821-42e0-a10b-a4b72b32d9f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:06.909774 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81326186-4821-42e0-a10b-a4b72b32d9f7 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:06.910603 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cce5d2a-39ae-4494-b56b-b372e03fe528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:06.911885 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce5d2a-39ae-4494-b56b-b372e03fe528 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:06.912447 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce5d2a-39ae-4494-b56b-b372e03fe528 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:06.913082 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce5d2a-39ae-4494-b56b-b372e03fe528 admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:06.916602 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cce5d2a-39ae-4494-b56b-b372e03fe528 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:06.919821 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cce5d2a-39ae-4494-b56b-b372e03fe528 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:06.939603 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1015/2026] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:15:06 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 12:15:06.944168 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81326186-4821-42e0-a10b-a4b72b32d9f7 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:06.972085 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81326186-4821-42e0-a10b-a4b72b32d9f7 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:06.980826 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa3702b-c090-4b39-8278-d969f3444c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:06.983248 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1012/2027] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:15:06 2022] POST /identity/v3/projects => generated 381 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:15:06.983874 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa3702b-c090-4b39-8278-d969f3444c39 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:06.984193 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa3702b-c090-4b39-8278-d969f3444c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:06.984449 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa3702b-c090-4b39-8278-d969f3444c39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:06.985167 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa3702b-c090-4b39-8278-d969f3444c39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:06.986316 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cce5d2a-39ae-4494-b56b-b372e03fe528 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:06.987698 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa3702b-c090-4b39-8278-d969f3444c39 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:07.002793 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1016/2028] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:15:06 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:15:07.008181 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa3702b-c090-4b39-8278-d969f3444c39 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:07.028469 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa3702b-c090-4b39-8278-d969f3444c39 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:07.084827 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cce5d2a-39ae-4494-b56b-b372e03fe528 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:07.091977 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be8389f-87b6-436a-9763-7fc8351e549f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:07.093133 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be8389f-87b6-436a-9763-7fc8351e549f admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:07.093133 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be8389f-87b6-436a-9763-7fc8351e549f admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:07.093312 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be8389f-87b6-436a-9763-7fc8351e549f admin admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:07.094021 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be8389f-87b6-436a-9763-7fc8351e549f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:07.095970 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be8389f-87b6-436a-9763-7fc8351e549f admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:07.103652 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aee09ff-b96e-4218-9fa9-b2ea7237eda7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:07.104426 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee09ff-b96e-4218-9fa9-b2ea7237eda7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:07.104777 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee09ff-b96e-4218-9fa9-b2ea7237eda7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:07.105125 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee09ff-b96e-4218-9fa9-b2ea7237eda7 admin admin] PATH_INFO: `/v3/projects/591e422446444d99b36966f117b36694/users/3cd11892ccf842c4bf24a8821f3005aa/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:07.108501 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:07.125067 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aee09ff-b96e-4218-9fa9-b2ea7237eda7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=591e422446444d99b36966f117b36694, user_id=3cd11892ccf842c4bf24a8821f3005aa, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:07.127771 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1013/2029] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:15:06 2022] POST /identity/v3/users => generated 291 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:15:07.128253 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aee09ff-b96e-4218-9fa9-b2ea7237eda7 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:07.128629 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be8389f-87b6-436a-9763-7fc8351e549f admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:07.131111 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-0be8389f-87b6-436a-9763-7fc8351e549f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:07.152795 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1017/2030] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:15:07 2022] PUT /identity/v3/projects/591e422446444d99b36966f117b36694/users/3cd11892ccf842c4bf24a8821f3005aa/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:07.152795 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aee09ff-b96e-4218-9fa9-b2ea7237eda7 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:07.153112 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be8389f-87b6-436a-9763-7fc8351e549f admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:07.229044 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d778ba1-4368-4216-bc88-ea041c7c4f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:07.230304 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d778ba1-4368-4216-bc88-ea041c7c4f88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:07.230785 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d778ba1-4368-4216-bc88-ea041c7c4f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:07.231058 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d778ba1-4368-4216-bc88-ea041c7c4f88 admin admin] PATH_INFO: `/v3/users/1d248594dc5a49dfb10c074d34947762` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:07.232497 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d778ba1-4368-4216-bc88-ea041c7c4f88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d248594dc5a49dfb10c074d34947762)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:07.232611 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aee09ff-b96e-4218-9fa9-b2ea7237eda7 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:07.234817 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d778ba1-4368-4216-bc88-ea041c7c4f88 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:07.238405 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6760d5-b260-4bd7-8f01-d037687be156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:07.239100 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6760d5-b260-4bd7-8f01-d037687be156 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:07.239383 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6760d5-b260-4bd7-8f01-d037687be156 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:07.239682 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6760d5-b260-4bd7-8f01-d037687be156 admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:07.240305 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6760d5-b260-4bd7-8f01-d037687be156 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:07.243596 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6760d5-b260-4bd7-8f01-d037687be156 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:07.253762 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1018/2031] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:15:07 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:15:07.254927 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c800f94-ad86-413c-ac59-d9acb74f018b None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:07.255264 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c800f94-ad86-413c-ac59-d9acb74f018b None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:07.255510 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c800f94-ad86-413c-ac59-d9acb74f018b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:07.287328 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-2c800f94-ad86-413c-ac59-d9acb74f018b None None] MFA Rules not processed for user `3cd11892ccf842c4bf24a8821f3005aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:07.289064 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-2c800f94-ad86-413c-ac59-d9acb74f018b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:07.298171 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-6d778ba1-4368-4216-bc88-ea041c7c4f88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d248594dc5a49dfb10c074d34947762'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:07.309640 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-6d778ba1-4368-4216-bc88-ea041c7c4f88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d248594dc5a49dfb10c074d34947762'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:07.319187 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-6d778ba1-4368-4216-bc88-ea041c7c4f88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d248594dc5a49dfb10c074d34947762'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:07.320814 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-6d778ba1-4368-4216-bc88-ea041c7c4f88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d248594dc5a49dfb10c074d34947762'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:07.346051 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-6d778ba1-4368-4216-bc88-ea041c7c4f88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d248594dc5a49dfb10c074d34947762'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:07.351946 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:07.355070 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1014/2032] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:15:07 2022] DELETE /identity/v3/users/1d248594dc5a49dfb10c074d34947762 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:07.356057 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d778ba1-4368-4216-bc88-ea041c7c4f88 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:07.357390 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-6d778ba1-4368-4216-bc88-ea041c7c4f88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:07.384355 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1019/2033] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:15:07.385825 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0551e6-589d-4151-b279-d0881d022bf1 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:07.386181 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0551e6-589d-4151-b279-d0881d022bf1 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:07.386768 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0551e6-589d-4151-b279-d0881d022bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:07.408249 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-0c0551e6-589d-4151-b279-d0881d022bf1 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:07.409658 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-0c0551e6-589d-4151-b279-d0881d022bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:07.453543 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d778ba1-4368-4216-bc88-ea041c7c4f88 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:07.459333 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c7b90c-793a-4040-af0f-5d76ec690807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:07.459506 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c7b90c-793a-4040-af0f-5d76ec690807 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:07.459843 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c7b90c-793a-4040-af0f-5d76ec690807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:07.460813 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c7b90c-793a-4040-af0f-5d76ec690807 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:07.461473 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c7b90c-793a-4040-af0f-5d76ec690807 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:07.463218 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c7b90c-793a-4040-af0f-5d76ec690807 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:07.487754 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1015/2034] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:15:07 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:15:07.488980 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c7b90c-793a-4040-af0f-5d76ec690807 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:07.491073 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-e5c7b90c-793a-4040-af0f-5d76ec690807 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:07.505263 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c7b90c-793a-4040-af0f-5d76ec690807 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:07.509388 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1020/2035] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:15:07.510203 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0551e6-589d-4151-b279-d0881d022bf1 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:07.513692 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7332b7dd-6679-49cf-b205-17ec5d01fce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:07.514317 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332b7dd-6679-49cf-b205-17ec5d01fce9 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:07.514651 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332b7dd-6679-49cf-b205-17ec5d01fce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:07.514932 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332b7dd-6679-49cf-b205-17ec5d01fce9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:07.516162 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7332b7dd-6679-49cf-b205-17ec5d01fce9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:07.519400 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7332b7dd-6679-49cf-b205-17ec5d01fce9 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:07.520494 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0551e6-589d-4151-b279-d0881d022bf1 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:07.525726 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13a58b6-a3eb-4a39-86dc-193c4ac7ed36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:07.526387 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a58b6-a3eb-4a39-86dc-193c4ac7ed36 admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:07.526670 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a58b6-a3eb-4a39-86dc-193c4ac7ed36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:07.526945 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a58b6-a3eb-4a39-86dc-193c4ac7ed36 admin admin] PATH_INFO: `/v3/projects/4585ed3a0f4b4aea8ab4b3cfaccc93cb/users/0dee81515d584b90a3490d4dcd61a09d/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:07.530802 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:07.544246 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13a58b6-a3eb-4a39-86dc-193c4ac7ed36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4585ed3a0f4b4aea8ab4b3cfaccc93cb, user_id=0dee81515d584b90a3490d4dcd61a09d, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:07.547017 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13a58b6-a3eb-4a39-86dc-193c4ac7ed36 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:07.563697 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1016/2036] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:15:07 2022] POST /identity/v3/projects => generated 385 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:15:07.565342 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1021/2037] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:15:07 2022] PUT /identity/v3/projects/4585ed3a0f4b4aea8ab4b3cfaccc93cb/users/0dee81515d584b90a3490d4dcd61a09d/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:07.567789 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7332b7dd-6679-49cf-b205-17ec5d01fce9 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:07.569147 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13a58b6-a3eb-4a39-86dc-193c4ac7ed36 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:07.570742 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-7332b7dd-6679-49cf-b205-17ec5d01fce9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:07.649525 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7332b7dd-6679-49cf-b205-17ec5d01fce9 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:07.667336 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13a58b6-a3eb-4a39-86dc-193c4ac7ed36 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:07.673127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47765da-a1dd-45f9-86c7-e2dbc8d8d478 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:07.673893 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47765da-a1dd-45f9-86c7-e2dbc8d8d478 service neutron] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:07.674178 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47765da-a1dd-45f9-86c7-e2dbc8d8d478 service neutron] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:07.674453 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47765da-a1dd-45f9-86c7-e2dbc8d8d478 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:07.758158 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47765da-a1dd-45f9-86c7-e2dbc8d8d478 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:07.760112 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4574a958-ffd4-4d1a-b4a3-9d0cecbe1333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:07.760851 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4574a958-ffd4-4d1a-b4a3-9d0cecbe1333 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:07.761140 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47765da-a1dd-45f9-86c7-e2dbc8d8d478 service neutron] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:07.761255 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4574a958-ffd4-4d1a-b4a3-9d0cecbe1333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:07.761578 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4574a958-ffd4-4d1a-b4a3-9d0cecbe1333 admin admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:07.762322 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4574a958-ffd4-4d1a-b4a3-9d0cecbe1333 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:07.764450 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4574a958-ffd4-4d1a-b4a3-9d0cecbe1333 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:07.775093 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1017/2038] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:15:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:15:07.776622 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc99d233-687e-401c-9355-c048be4d3d35 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:07.777056 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc99d233-687e-401c-9355-c048be4d3d35 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:07.777474 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc99d233-687e-401c-9355-c048be4d3d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:07.793473 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:07.806227 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1022/2039] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:15:07 2022] POST /identity/v3/users => generated 299 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:15:07.808132 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4574a958-ffd4-4d1a-b4a3-9d0cecbe1333 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:07.815807 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-fc99d233-687e-401c-9355-c048be4d3d35 None None] MFA Rules not processed for user `0dee81515d584b90a3490d4dcd61a09d`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:07.817675 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-fc99d233-687e-401c-9355-c048be4d3d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:07.829035 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4574a958-ffd4-4d1a-b4a3-9d0cecbe1333 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:07.837178 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222c71e0-c055-46e5-9443-e7cb0a01a1ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:07.837990 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222c71e0-c055-46e5-9443-e7cb0a01a1ed admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:07.838399 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222c71e0-c055-46e5-9443-e7cb0a01a1ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:07.840707 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222c71e0-c055-46e5-9443-e7cb0a01a1ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:07.842293 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222c71e0-c055-46e5-9443-e7cb0a01a1ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:07.845589 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222c71e0-c055-46e5-9443-e7cb0a01a1ed admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:07.859155 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1023/2040] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:15:07 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 12:15:07.863840 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222c71e0-c055-46e5-9443-e7cb0a01a1ed admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:07.879838 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222c71e0-c055-46e5-9443-e7cb0a01a1ed admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:07.890983 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6a7002-b093-42fd-b030-2b8929d7ca4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:07.891959 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a7002-b093-42fd-b030-2b8929d7ca4d admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:07.892346 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a7002-b093-42fd-b030-2b8929d7ca4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:07.892729 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a7002-b093-42fd-b030-2b8929d7ca4d admin admin] PATH_INFO: `/v3/projects/daff1723206944e8a25eb146224e9c2b/users/55ef532a0076465fad6145d33a85b1b3/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:07.920848 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6a7002-b093-42fd-b030-2b8929d7ca4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=daff1723206944e8a25eb146224e9c2b, user_id=55ef532a0076465fad6145d33a85b1b3, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:07.925880 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6a7002-b093-42fd-b030-2b8929d7ca4d admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:07.930945 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1018/2041] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:07 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:15:07.935855 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc99d233-687e-401c-9355-c048be4d3d35 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:07.956397 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1024/2042] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:15:07 2022] PUT /identity/v3/projects/daff1723206944e8a25eb146224e9c2b/users/55ef532a0076465fad6145d33a85b1b3/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 12:15:07.958030 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6a7002-b093-42fd-b030-2b8929d7ca4d admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:07.962165 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc99d233-687e-401c-9355-c048be4d3d35 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:08.005577 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:08.070130 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bc73b7-9972-4b58-8f5b-3ed345d9500d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:08.071881 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc73b7-9972-4b58-8f5b-3ed345d9500d admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:08.075323 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc73b7-9972-4b58-8f5b-3ed345d9500d admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:08.075323 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc73b7-9972-4b58-8f5b-3ed345d9500d admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:08.075323 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bc73b7-9972-4b58-8f5b-3ed345d9500d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:08.082740 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bc73b7-9972-4b58-8f5b-3ed345d9500d admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:08.101827 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6a7002-b093-42fd-b030-2b8929d7ca4d admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:08.117438 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f01cfeb-3966-486f-b45a-0f799ca3011d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:08.119291 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f01cfeb-3966-486f-b45a-0f799ca3011d admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:08.119585 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f01cfeb-3966-486f-b45a-0f799ca3011d admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:08.119861 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f01cfeb-3966-486f-b45a-0f799ca3011d admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:08.120517 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f01cfeb-3966-486f-b45a-0f799ca3011d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:08.129044 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f01cfeb-3966-486f-b45a-0f799ca3011d admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:08.144002 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1019/2043] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:15:07 2022] POST /identity/v3/projects => generated 379 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:15:08.144207 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1025/2044] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:15:07 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:15:08.149424 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bc73b7-9972-4b58-8f5b-3ed345d9500d admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:08.151036 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f01cfeb-3966-486f-b45a-0f799ca3011d admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:08.168225 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f01cfeb-3966-486f-b45a-0f799ca3011d admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:08.244438 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658c25a8-f435-4e00-99e7-3f3a697cd951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:08.245098 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658c25a8-f435-4e00-99e7-3f3a697cd951 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:08.245313 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658c25a8-f435-4e00-99e7-3f3a697cd951 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:08.245576 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658c25a8-f435-4e00-99e7-3f3a697cd951 admin admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:08.246207 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658c25a8-f435-4e00-99e7-3f3a697cd951 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:08.246721 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bc73b7-9972-4b58-8f5b-3ed345d9500d admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:08.248551 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658c25a8-f435-4e00-99e7-3f3a697cd951 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:08.252456 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c002a6a3-fea8-4188-8f77-f4d981b19cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:08.253334 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c002a6a3-fea8-4188-8f77-f4d981b19cc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:08.253689 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c002a6a3-fea8-4188-8f77-f4d981b19cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:08.254035 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c002a6a3-fea8-4188-8f77-f4d981b19cc1 admin admin] PATH_INFO: `/v3/projects/daff1723206944e8a25eb146224e9c2b/users/55ef532a0076465fad6145d33a85b1b3/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:08.258184 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c002a6a3-fea8-4188-8f77-f4d981b19cc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=daff1723206944e8a25eb146224e9c2b, user_id=55ef532a0076465fad6145d33a85b1b3, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:08.263352 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c002a6a3-fea8-4188-8f77-f4d981b19cc1 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:08.279310 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1026/2046] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:15:08 2022] POST /identity/v3/users => generated 296 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 12:15:08.280572 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658c25a8-f435-4e00-99e7-3f3a697cd951 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:08.285949 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1020/2046] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:15:08 2022] PUT /identity/v3/projects/daff1723206944e8a25eb146224e9c2b/users/55ef532a0076465fad6145d33a85b1b3/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:08.287446 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c62410-e81b-449b-a277-34adfdba4b2d None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:08.287735 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c62410-e81b-449b-a277-34adfdba4b2d None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:08.288012 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c62410-e81b-449b-a277-34adfdba4b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:08.299146 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658c25a8-f435-4e00-99e7-3f3a697cd951 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:08.329926 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-57c62410-e81b-449b-a277-34adfdba4b2d None None] MFA Rules not processed for user `55ef532a0076465fad6145d33a85b1b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:08.331389 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-57c62410-e81b-449b-a277-34adfdba4b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:08.378216 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:08.399011 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac0e2c7-a5c2-469a-ad8c-6654219bf2d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:08.399819 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac0e2c7-a5c2-469a-ad8c-6654219bf2d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:08.402390 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac0e2c7-a5c2-469a-ad8c-6654219bf2d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:08.403051 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac0e2c7-a5c2-469a-ad8c-6654219bf2d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:08.405564 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac0e2c7-a5c2-469a-ad8c-6654219bf2d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:08.407834 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac0e2c7-a5c2-469a-ad8c-6654219bf2d0 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:08.423964 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1027/2047] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:15:08.425573 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac0e2c7-a5c2-469a-ad8c-6654219bf2d0 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:08.428450 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-7ac0e2c7-a5c2-469a-ad8c-6654219bf2d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:08.442994 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac0e2c7-a5c2-469a-ad8c-6654219bf2d0 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:08.447104 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1021/2048] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:08 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:15:08.449348 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9959469-a8c0-4e00-9300-a40f71338a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:08.449994 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9959469-a8c0-4e00-9300-a40f71338a4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:08.450279 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9959469-a8c0-4e00-9300-a40f71338a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:08.450555 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9959469-a8c0-4e00-9300-a40f71338a4e admin admin] PATH_INFO: `/v3/projects/78ba2c038ceb46b9b5bbf241b2a55a1a` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:08.451754 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9959469-a8c0-4e00-9300-a40f71338a4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=78ba2c038ceb46b9b5bbf241b2a55a1a)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:08.453168 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c62410-e81b-449b-a277-34adfdba4b2d None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:08.453776 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9959469-a8c0-4e00-9300-a40f71338a4e admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:08.465594 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c62410-e81b-449b-a277-34adfdba4b2d None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:08.470959 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea440fd-3df3-40a6-872b-d373de99942d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:08.471664 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea440fd-3df3-40a6-872b-d373de99942d admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:08.471945 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea440fd-3df3-40a6-872b-d373de99942d admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:08.472252 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea440fd-3df3-40a6-872b-d373de99942d admin admin] PATH_INFO: `/v3/projects/ff1bfe6f011143e7a460256e28ef43ce/users/e03b5fbc07774d43a5344561bbc87ecc/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:08.493251 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea440fd-3df3-40a6-872b-d373de99942d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff1bfe6f011143e7a460256e28ef43ce, user_id=e03b5fbc07774d43a5344561bbc87ecc, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:08.496131 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea440fd-3df3-40a6-872b-d373de99942d admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:08.516356 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-b9959469-a8c0-4e00-9300-a40f71338a4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78ba2c038ceb46b9b5bbf241b2a55a1a', 'request_id': 'req-b9959469-a8c0-4e00-9300-a40f71338a4e'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:08.517880 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1022/2049] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:15:08 2022] PUT /identity/v3/projects/ff1bfe6f011143e7a460256e28ef43ce/users/e03b5fbc07774d43a5344561bbc87ecc/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:08.518875 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea440fd-3df3-40a6-872b-d373de99942d admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:08.520760 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-eea440fd-3df3-40a6-872b-d373de99942d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:08.540524 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-b9959469-a8c0-4e00-9300-a40f71338a4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78ba2c038ceb46b9b5bbf241b2a55a1a', 'request_id': 'req-b9959469-a8c0-4e00-9300-a40f71338a4e'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:08.547250 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:08.558176 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-b9959469-a8c0-4e00-9300-a40f71338a4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78ba2c038ceb46b9b5bbf241b2a55a1a', 'request_id': 'req-b9959469-a8c0-4e00-9300-a40f71338a4e'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:08.581111 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-b9959469-a8c0-4e00-9300-a40f71338a4e admin admin] The token cache is being invalidate because project 78ba2c038ceb46b9b5bbf241b2a55a1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81712) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:15:08.581416 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-b9959469-a8c0-4e00-9300-a40f71338a4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:08.582505 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-b9959469-a8c0-4e00-9300-a40f71338a4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:08.587026 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1028/2050] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:15:08 2022] DELETE /identity/v3/projects/78ba2c038ceb46b9b5bbf241b2a55a1a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:08.587854 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9959469-a8c0-4e00-9300-a40f71338a4e admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:08.589656 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-b9959469-a8c0-4e00-9300-a40f71338a4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:08.633484 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea440fd-3df3-40a6-872b-d373de99942d admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:08.638354 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7e87b6-754b-46d3-a2c6-a74cf4f644d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:08.639391 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e87b6-754b-46d3-a2c6-a74cf4f644d1 service cinder] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:08.639776 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e87b6-754b-46d3-a2c6-a74cf4f644d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:08.640190 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7e87b6-754b-46d3-a2c6-a74cf4f644d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:08.643570 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-ff7e87b6-754b-46d3-a2c6-a74cf4f644d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:08.674119 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9959469-a8c0-4e00-9300-a40f71338a4e admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:08.724729 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7e87b6-754b-46d3-a2c6-a74cf4f644d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:08.727388 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7e87b6-754b-46d3-a2c6-a74cf4f644d1 service cinder] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:08.740156 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1023/2051] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:15:08 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:15:08.742300 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7e87b6-754b-46d3-a2c6-a74cf4f644d1 service cinder] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:08.747596 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-ff7e87b6-754b-46d3-a2c6-a74cf4f644d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:08.762766 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d14757-fb3c-4d14-b17f-5bfa4e9ba151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:08.764005 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d14757-fb3c-4d14-b17f-5bfa4e9ba151 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:08.765608 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d14757-fb3c-4d14-b17f-5bfa4e9ba151 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:08.766182 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d14757-fb3c-4d14-b17f-5bfa4e9ba151 admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:08.767502 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d14757-fb3c-4d14-b17f-5bfa4e9ba151 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:08.770186 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d14757-fb3c-4d14-b17f-5bfa4e9ba151 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:08.785749 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1029/2052] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:15:08.790169 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d14757-fb3c-4d14-b17f-5bfa4e9ba151 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:08.850148 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d14757-fb3c-4d14-b17f-5bfa4e9ba151 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:08.862996 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08e420a-06da-4305-be3a-6ccb25580e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:08.865746 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e420a-06da-4305-be3a-6ccb25580e89 admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:08.866156 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e420a-06da-4305-be3a-6ccb25580e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:08.866737 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e420a-06da-4305-be3a-6ccb25580e89 admin admin] PATH_INFO: `/v3/projects/ff1bfe6f011143e7a460256e28ef43ce/users/e03b5fbc07774d43a5344561bbc87ecc/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:08.884960 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08e420a-06da-4305-be3a-6ccb25580e89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff1bfe6f011143e7a460256e28ef43ce, user_id=e03b5fbc07774d43a5344561bbc87ecc, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:08.888675 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08e420a-06da-4305-be3a-6ccb25580e89 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:08.896309 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:08.918687 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1030/2053] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:15:08 2022] PUT /identity/v3/projects/ff1bfe6f011143e7a460256e28ef43ce/users/e03b5fbc07774d43a5344561bbc87ecc/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:08.920578 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee14c80-2245-4b30-b11c-a7672e791065 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:08.920873 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee14c80-2245-4b30-b11c-a7672e791065 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:08.921173 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee14c80-2245-4b30-b11c-a7672e791065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:08.921394 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7e87b6-754b-46d3-a2c6-a74cf4f644d1 service cinder] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:08.972483 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-5ee14c80-2245-4b30-b11c-a7672e791065 None None] MFA Rules not processed for user `e03b5fbc07774d43a5344561bbc87ecc`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:08.974040 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-5ee14c80-2245-4b30-b11c-a7672e791065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:09.006397 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07abd1ab-c81b-49c6-a634-3ed735605e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:09.007856 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07abd1ab-c81b-49c6-a634-3ed735605e27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:09.008148 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07abd1ab-c81b-49c6-a634-3ed735605e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:09.008480 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07abd1ab-c81b-49c6-a634-3ed735605e27 admin admin] PATH_INFO: `/v3/users/40ed2a2a4ea34a73b778b979f7c06b66` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:09.010393 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07abd1ab-c81b-49c6-a634-3ed735605e27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40ed2a2a4ea34a73b778b979f7c06b66)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:09.013154 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07abd1ab-c81b-49c6-a634-3ed735605e27 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:09.031939 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:09.084810 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1031/2054] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:08 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:15:09.094909 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-07abd1ab-c81b-49c6-a634-3ed735605e27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40ed2a2a4ea34a73b778b979f7c06b66'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:09.097637 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee14c80-2245-4b30-b11c-a7672e791065 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:09.100073 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-5ee14c80-2245-4b30-b11c-a7672e791065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:09.122129 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-07abd1ab-c81b-49c6-a634-3ed735605e27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40ed2a2a4ea34a73b778b979f7c06b66'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:09.133507 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-07abd1ab-c81b-49c6-a634-3ed735605e27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40ed2a2a4ea34a73b778b979f7c06b66'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:09.136742 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-07abd1ab-c81b-49c6-a634-3ed735605e27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40ed2a2a4ea34a73b778b979f7c06b66'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:09.162015 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-07abd1ab-c81b-49c6-a634-3ed735605e27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40ed2a2a4ea34a73b778b979f7c06b66'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:09.177174 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1024/2055] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:15:08 2022] DELETE /identity/v3/users/40ed2a2a4ea34a73b778b979f7c06b66 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:09.221981 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee14c80-2245-4b30-b11c-a7672e791065 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:09.314314 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333c32a4-fb8e-49a6-b8dd-d610e8096105 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:09.315075 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333c32a4-fb8e-49a6-b8dd-d610e8096105 service cinder] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:09.315302 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333c32a4-fb8e-49a6-b8dd-d610e8096105 service cinder] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:09.315530 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333c32a4-fb8e-49a6-b8dd-d610e8096105 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:09.317578 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-333c32a4-fb8e-49a6-b8dd-d610e8096105 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:09.420993 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:09.458850 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333c32a4-fb8e-49a6-b8dd-d610e8096105 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:09.464928 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333c32a4-fb8e-49a6-b8dd-d610e8096105 service cinder] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:09.480934 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1032/2056] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:15:09 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 12:15:09.943488 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07abd1ab-c81b-49c6-a634-3ed735605e27 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:09.945666 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-07abd1ab-c81b-49c6-a634-3ed735605e27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:10.108016 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07abd1ab-c81b-49c6-a634-3ed735605e27 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:10.113081 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bde634-fb12-4104-a73f-88156e8b13bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:10.113753 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bde634-fb12-4104-a73f-88156e8b13bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:10.114034 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bde634-fb12-4104-a73f-88156e8b13bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:10.114308 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bde634-fb12-4104-a73f-88156e8b13bb admin admin] PATH_INFO: `/v3/projects/d8522abdf0774a7993fc55c493ccfc8d` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:10.115593 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bde634-fb12-4104-a73f-88156e8b13bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8522abdf0774a7993fc55c493ccfc8d)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:10.118751 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bde634-fb12-4104-a73f-88156e8b13bb admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:10.292022 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-f4bde634-fb12-4104-a73f-88156e8b13bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8522abdf0774a7993fc55c493ccfc8d', 'request_id': 'req-f4bde634-fb12-4104-a73f-88156e8b13bb'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:10.307949 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-f4bde634-fb12-4104-a73f-88156e8b13bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8522abdf0774a7993fc55c493ccfc8d', 'request_id': 'req-f4bde634-fb12-4104-a73f-88156e8b13bb'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:10.322154 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-f4bde634-fb12-4104-a73f-88156e8b13bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8522abdf0774a7993fc55c493ccfc8d', 'request_id': 'req-f4bde634-fb12-4104-a73f-88156e8b13bb'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:10.339826 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-f4bde634-fb12-4104-a73f-88156e8b13bb admin admin] The token cache is being invalidate because project d8522abdf0774a7993fc55c493ccfc8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:15:10.340123 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-f4bde634-fb12-4104-a73f-88156e8b13bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:10.341380 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-f4bde634-fb12-4104-a73f-88156e8b13bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:10.343433 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1025/2057] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:15:09 2022] DELETE /identity/v3/projects/d8522abdf0774a7993fc55c493ccfc8d => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:10.348605 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22cdaf7-fef7-4f91-b216-8c3cada3b8e4 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:10.349062 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22cdaf7-fef7-4f91-b216-8c3cada3b8e4 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:10.349062 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22cdaf7-fef7-4f91-b216-8c3cada3b8e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:10.388504 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-e22cdaf7-fef7-4f91-b216-8c3cada3b8e4 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:10.389707 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-e22cdaf7-fef7-4f91-b216-8c3cada3b8e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:10.498296 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bde634-fb12-4104-a73f-88156e8b13bb admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:10.499941 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-f4bde634-fb12-4104-a73f-88156e8b13bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:10.532799 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1033/2058] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:15:10.541377 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2bd8ed-4370-4c58-b6ef-cb8415013e56 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:10.542523 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2bd8ed-4370-4c58-b6ef-cb8415013e56 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:10.543041 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2bd8ed-4370-4c58-b6ef-cb8415013e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:10.550349 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:10.591036 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-bd2bd8ed-4370-4c58-b6ef-cb8415013e56 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:10.593140 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-bd2bd8ed-4370-4c58-b6ef-cb8415013e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:10.624988 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1034/2059] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:15:10.630549 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2bd8ed-4370-4c58-b6ef-cb8415013e56 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:10.647219 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2bd8ed-4370-4c58-b6ef-cb8415013e56 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:10.654317 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bde634-fb12-4104-a73f-88156e8b13bb admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:10.654582 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7415afb7-5d19-4db6-a31a-2c3faf192f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:10.655436 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7415afb7-5d19-4db6-a31a-2c3faf192f48 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:10.656304 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7415afb7-5d19-4db6-a31a-2c3faf192f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:10.656579 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7415afb7-5d19-4db6-a31a-2c3faf192f48 admin admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:10.657242 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7415afb7-5d19-4db6-a31a-2c3faf192f48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:15:10.657547 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7415afb7-5d19-4db6-a31a-2c3faf192f48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:10.659182 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7415afb7-5d19-4db6-a31a-2c3faf192f48 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:10.685507 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a768543d-ad74-4e54-b102-fb56dbc6f7a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:10.685735 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1035/2060] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:15:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:15:10.686182 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a768543d-ad74-4e54-b102-fb56dbc6f7a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:10.686182 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a768543d-ad74-4e54-b102-fb56dbc6f7a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:10.686182 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a768543d-ad74-4e54-b102-fb56dbc6f7a2 admin admin] PATH_INFO: `/v3/users/0dee81515d584b90a3490d4dcd61a09d` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:10.687403 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7415afb7-5d19-4db6-a31a-2c3faf192f48 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:10.694176 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a768543d-ad74-4e54-b102-fb56dbc6f7a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0dee81515d584b90a3490d4dcd61a09d)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:10.697575 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a768543d-ad74-4e54-b102-fb56dbc6f7a2 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:10.706394 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7415afb7-5d19-4db6-a31a-2c3faf192f48 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:10.728371 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e1a96f-dde9-4967-b59f-eab1e35647fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:10.728985 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e1a96f-dde9-4967-b59f-eab1e35647fc admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:10.729265 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e1a96f-dde9-4967-b59f-eab1e35647fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:10.729719 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e1a96f-dde9-4967-b59f-eab1e35647fc admin admin] PATH_INFO: `/v3/projects` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:10.730799 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e1a96f-dde9-4967-b59f-eab1e35647fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:10.732714 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e1a96f-dde9-4967-b59f-eab1e35647fc admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:10.782762 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1036/2061] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:15:10 2022] POST /identity/v3/projects => generated 395 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:15:10.784107 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e1a96f-dde9-4967-b59f-eab1e35647fc admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:10.785892 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-43e1a96f-dde9-4967-b59f-eab1e35647fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:10.810223 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-a768543d-ad74-4e54-b102-fb56dbc6f7a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0dee81515d584b90a3490d4dcd61a09d'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:10.822063 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-a768543d-ad74-4e54-b102-fb56dbc6f7a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0dee81515d584b90a3490d4dcd61a09d'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:10.835987 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-a768543d-ad74-4e54-b102-fb56dbc6f7a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0dee81515d584b90a3490d4dcd61a09d'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:10.837392 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-a768543d-ad74-4e54-b102-fb56dbc6f7a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0dee81515d584b90a3490d4dcd61a09d'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:10.853094 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-a768543d-ad74-4e54-b102-fb56dbc6f7a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0dee81515d584b90a3490d4dcd61a09d'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:10.857617 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e1a96f-dde9-4967-b59f-eab1e35647fc admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:10.863510 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1026/2062] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:15:10 2022] DELETE /identity/v3/users/0dee81515d584b90a3490d4dcd61a09d => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 12:15:10.863510 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a768543d-ad74-4e54-b102-fb56dbc6f7a2 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:10.863510 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-a768543d-ad74-4e54-b102-fb56dbc6f7a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:10.864608 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c4de9f-4cb7-41c0-98df-98e27757b126 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:10.865349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4de9f-4cb7-41c0-98df-98e27757b126 service cinder] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:10.865685 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4de9f-4cb7-41c0-98df-98e27757b126 service cinder] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:10.866134 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4de9f-4cb7-41c0-98df-98e27757b126 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:10.869177 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-b4c4de9f-4cb7-41c0-98df-98e27757b126 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:10.949839 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c4de9f-4cb7-41c0-98df-98e27757b126 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:10.951852 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c4de9f-4cb7-41c0-98df-98e27757b126 service cinder] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:10.963184 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1037/2063] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:15:10 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:15:10.965219 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51474ef-dbd3-4d69-b8e4-9fe4b394f810 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:10.965219 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51474ef-dbd3-4d69-b8e4-9fe4b394f810 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:10.965480 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51474ef-dbd3-4d69-b8e4-9fe4b394f810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:10.969796 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a768543d-ad74-4e54-b102-fb56dbc6f7a2 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:10.973003 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:10.980159 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8c624c-43a7-4262-be19-642844127bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:10.981408 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8c624c-43a7-4262-be19-642844127bfa admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:10.981909 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8c624c-43a7-4262-be19-642844127bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:10.982382 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8c624c-43a7-4262-be19-642844127bfa admin admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:10.983245 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8c624c-43a7-4262-be19-642844127bfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:10.986112 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8c624c-43a7-4262-be19-642844127bfa admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:11.001868 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-d51474ef-dbd3-4d69-b8e4-9fe4b394f810 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:11.003759 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-d51474ef-dbd3-4d69-b8e4-9fe4b394f810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:11.015418 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1038/2064] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:15:11.021763 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1027/2065] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:15:10 2022] POST /identity/v3/users => generated 298 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:15:11.028307 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8c624c-43a7-4262-be19-642844127bfa admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:11.028651 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51474ef-dbd3-4d69-b8e4-9fe4b394f810 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:11.030557 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-6b8c624c-43a7-4262-be19-642844127bfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:11.082470 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51474ef-dbd3-4d69-b8e4-9fe4b394f810 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:11.094811 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:11.098798 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f3f98c-ec5c-404e-a11d-9179d5a5150b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:11.100227 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f3f98c-ec5c-404e-a11d-9179d5a5150b admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:11.101525 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f3f98c-ec5c-404e-a11d-9179d5a5150b admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:11.101741 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f3f98c-ec5c-404e-a11d-9179d5a5150b admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:11.104510 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f3f98c-ec5c-404e-a11d-9179d5a5150b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:11.108938 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f3f98c-ec5c-404e-a11d-9179d5a5150b admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:11.137896 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1039/2066] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:15:11.143470 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f3f98c-ec5c-404e-a11d-9179d5a5150b admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:11.144462 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8c624c-43a7-4262-be19-642844127bfa admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:11.150385 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f671ae-be76-4db8-abec-a61717dfbd04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:11.151661 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f671ae-be76-4db8-abec-a61717dfbd04 service neutron] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:11.152332 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f671ae-be76-4db8-abec-a61717dfbd04 service neutron] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:11.152820 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f671ae-be76-4db8-abec-a61717dfbd04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:11.165998 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f671ae-be76-4db8-abec-a61717dfbd04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:11.167465 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f3f98c-ec5c-404e-a11d-9179d5a5150b admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:11.168295 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f671ae-be76-4db8-abec-a61717dfbd04 service neutron] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:11.180742 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1028/2067] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:15:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:15:11.186706 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3446c4-0a2c-430a-a41f-f339ad7f6d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:11.187803 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3446c4-0a2c-430a-a41f-f339ad7f6d85 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:11.188345 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3446c4-0a2c-430a-a41f-f339ad7f6d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:11.188851 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3446c4-0a2c-430a-a41f-f339ad7f6d85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:11.189952 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3446c4-0a2c-430a-a41f-f339ad7f6d85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:11.192045 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3446c4-0a2c-430a-a41f-f339ad7f6d85 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:11.208160 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1040/2068] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:15:11.214874 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f671ae-be76-4db8-abec-a61717dfbd04 service neutron] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:11.231465 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f671ae-be76-4db8-abec-a61717dfbd04 service neutron] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:11.248822 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50078f56-ee86-4ea1-ae6f-a02a68414c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:11.251416 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50078f56-ee86-4ea1-ae6f-a02a68414c41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:11.253192 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50078f56-ee86-4ea1-ae6f-a02a68414c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:11.253702 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50078f56-ee86-4ea1-ae6f-a02a68414c41 admin admin] PATH_INFO: `/v3/projects/2aa27dd0ad0941fea32762d13dca0212/users/0e77d04ed68e41ae8841f13419b87a84/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:11.285027 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50078f56-ee86-4ea1-ae6f-a02a68414c41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2aa27dd0ad0941fea32762d13dca0212, user_id=0e77d04ed68e41ae8841f13419b87a84, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:11.290459 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50078f56-ee86-4ea1-ae6f-a02a68414c41 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:11.319161 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1029/2069] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:15:11 2022] PUT /identity/v3/projects/2aa27dd0ad0941fea32762d13dca0212/users/0e77d04ed68e41ae8841f13419b87a84/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:11.330106 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5111b-03e6-44e5-bfba-201cef679972 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:11.330600 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5111b-03e6-44e5-bfba-201cef679972 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:11.331027 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5111b-03e6-44e5-bfba-201cef679972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:11.383918 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-3fb5111b-03e6-44e5-bfba-201cef679972 None None] MFA Rules not processed for user `0e77d04ed68e41ae8841f13419b87a84`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:11.385204 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-3fb5111b-03e6-44e5-bfba-201cef679972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:11.546013 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1041/2070] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:11 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:15:11.547761 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50078f56-ee86-4ea1-ae6f-a02a68414c41 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:11.699600 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50078f56-ee86-4ea1-ae6f-a02a68414c41 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:11.712786 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63086e4c-0e20-4b21-b25c-d8b748aada69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:11.713420 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63086e4c-0e20-4b21-b25c-d8b748aada69 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:11.713704 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63086e4c-0e20-4b21-b25c-d8b748aada69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:11.714606 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63086e4c-0e20-4b21-b25c-d8b748aada69 admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:11.715667 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63086e4c-0e20-4b21-b25c-d8b748aada69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:11.718877 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63086e4c-0e20-4b21-b25c-d8b748aada69 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:11.783470 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb5111b-03e6-44e5-bfba-201cef679972 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:11.785133 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-3fb5111b-03e6-44e5-bfba-201cef679972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:11.789807 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1030/2071] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:15:11 2022] POST /identity/v3/projects => generated 401 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:15:11.792775 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63086e4c-0e20-4b21-b25c-d8b748aada69 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:11.836396 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb5111b-03e6-44e5-bfba-201cef679972 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:11.890794 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63086e4c-0e20-4b21-b25c-d8b748aada69 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:11.905786 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28c81f4-8c4f-4dec-80da-da1db4c6a0ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:11.906412 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28c81f4-8c4f-4dec-80da-da1db4c6a0ac admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:11.906686 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28c81f4-8c4f-4dec-80da-da1db4c6a0ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:11.906956 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28c81f4-8c4f-4dec-80da-da1db4c6a0ac admin admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:11.907757 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28c81f4-8c4f-4dec-80da-da1db4c6a0ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:11.909844 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28c81f4-8c4f-4dec-80da-da1db4c6a0ac admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:11.955536 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1031/2072] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:15:11 2022] POST /identity/v3/users => generated 307 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:15:11.955536 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28c81f4-8c4f-4dec-80da-da1db4c6a0ac admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:11.971069 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28c81f4-8c4f-4dec-80da-da1db4c6a0ac admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:11.974285 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40095758-ed86-4ff9-8812-c14b1e9d7a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:11.974931 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40095758-ed86-4ff9-8812-c14b1e9d7a90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:11.975233 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40095758-ed86-4ff9-8812-c14b1e9d7a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:11.975570 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40095758-ed86-4ff9-8812-c14b1e9d7a90 admin admin] PATH_INFO: `/v3/projects/4585ed3a0f4b4aea8ab4b3cfaccc93cb` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:11.977703 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e7c4c-958d-4e11-aaba-fdfa3e1a8637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:11.978644 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e7c4c-958d-4e11-aaba-fdfa3e1a8637 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:11.979588 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e7c4c-958d-4e11-aaba-fdfa3e1a8637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:11.979588 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e7c4c-958d-4e11-aaba-fdfa3e1a8637 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:11.980158 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655e7c4c-958d-4e11-aaba-fdfa3e1a8637 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:11.980489 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40095758-ed86-4ff9-8812-c14b1e9d7a90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4585ed3a0f4b4aea8ab4b3cfaccc93cb)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:11.982063 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655e7c4c-958d-4e11-aaba-fdfa3e1a8637 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:11.982411 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40095758-ed86-4ff9-8812-c14b1e9d7a90 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:11.995494 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1032/2073] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 12:15:11.996291 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e7c4c-958d-4e11-aaba-fdfa3e1a8637 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:11.999676 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-655e7c4c-958d-4e11-aaba-fdfa3e1a8637 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:12.014866 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:12.064992 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-40095758-ed86-4ff9-8812-c14b1e9d7a90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4585ed3a0f4b4aea8ab4b3cfaccc93cb', 'request_id': 'req-40095758-ed86-4ff9-8812-c14b1e9d7a90'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:12.075440 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-40095758-ed86-4ff9-8812-c14b1e9d7a90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4585ed3a0f4b4aea8ab4b3cfaccc93cb', 'request_id': 'req-40095758-ed86-4ff9-8812-c14b1e9d7a90'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:12.085256 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-40095758-ed86-4ff9-8812-c14b1e9d7a90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4585ed3a0f4b4aea8ab4b3cfaccc93cb', 'request_id': 'req-40095758-ed86-4ff9-8812-c14b1e9d7a90'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:12.096406 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e7c4c-958d-4e11-aaba-fdfa3e1a8637 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:12.099259 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-40095758-ed86-4ff9-8812-c14b1e9d7a90 admin admin] The token cache is being invalidate because project 4585ed3a0f4b4aea8ab4b3cfaccc93cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81712) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:15:12.099770 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-40095758-ed86-4ff9-8812-c14b1e9d7a90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:12.100750 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-40095758-ed86-4ff9-8812-c14b1e9d7a90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:12.102900 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1042/2074] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:15:11 2022] DELETE /identity/v3/projects/4585ed3a0f4b4aea8ab4b3cfaccc93cb => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:12.102900 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:12.103975 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40095758-ed86-4ff9-8812-c14b1e9d7a90 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:12.105800 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-40095758-ed86-4ff9-8812-c14b1e9d7a90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:12.192782 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c3b94e-f352-4ba6-8565-f8bb7ba0560d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:12.193421 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c3b94e-f352-4ba6-8565-f8bb7ba0560d service cinder] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:12.195381 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c3b94e-f352-4ba6-8565-f8bb7ba0560d service cinder] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:12.195902 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c3b94e-f352-4ba6-8565-f8bb7ba0560d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:12.202421 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-74c3b94e-f352-4ba6-8565-f8bb7ba0560d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:12.204186 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40095758-ed86-4ff9-8812-c14b1e9d7a90 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:12.216421 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f479e3-2f48-46d8-8359-d93006c408fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:12.217466 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f479e3-2f48-46d8-8359-d93006c408fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:12.217758 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f479e3-2f48-46d8-8359-d93006c408fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:12.218070 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f479e3-2f48-46d8-8359-d93006c408fb admin admin] PATH_INFO: `/v3/projects/81f786893ba84b829555a35286638f0d/users/1ec92d8d8dae4c4d90ba2eef8ce60dab/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:12.236992 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f479e3-2f48-46d8-8359-d93006c408fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=81f786893ba84b829555a35286638f0d, user_id=1ec92d8d8dae4c4d90ba2eef8ce60dab, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:12.240435 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f479e3-2f48-46d8-8359-d93006c408fb admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:12.259116 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1043/2075] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:15:12 2022] PUT /identity/v3/projects/81f786893ba84b829555a35286638f0d/users/1ec92d8d8dae4c4d90ba2eef8ce60dab/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:12.262872 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f479e3-2f48-46d8-8359-d93006c408fb admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:12.264634 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-66f479e3-2f48-46d8-8359-d93006c408fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:12.290482 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c3b94e-f352-4ba6-8565-f8bb7ba0560d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:12.291881 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c3b94e-f352-4ba6-8565-f8bb7ba0560d service cinder] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:12.304160 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1033/2076] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:15:11 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:15:12.304750 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c3b94e-f352-4ba6-8565-f8bb7ba0560d service cinder] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:12.316175 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c3b94e-f352-4ba6-8565-f8bb7ba0560d service cinder] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:12.337609 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f479e3-2f48-46d8-8359-d93006c408fb admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:12.344450 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d6c625-cf2c-409d-b227-fd45b6f65d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:12.345516 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d6c625-cf2c-409d-b227-fd45b6f65d88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:12.346082 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d6c625-cf2c-409d-b227-fd45b6f65d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:12.346585 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d6c625-cf2c-409d-b227-fd45b6f65d88 admin admin] PATH_INFO: `/v3/users/e03b5fbc07774d43a5344561bbc87ecc` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:12.347997 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d6c625-cf2c-409d-b227-fd45b6f65d88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e03b5fbc07774d43a5344561bbc87ecc)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:12.351031 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d6c625-cf2c-409d-b227-fd45b6f65d88 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:12.404606 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e8a13e-9b67-4cf2-b7d9-10bcaab223d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:12.405318 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e8a13e-9b67-4cf2-b7d9-10bcaab223d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:12.405668 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e8a13e-9b67-4cf2-b7d9-10bcaab223d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:12.406022 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e8a13e-9b67-4cf2-b7d9-10bcaab223d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:12.406868 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e8a13e-9b67-4cf2-b7d9-10bcaab223d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:12.409541 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e8a13e-9b67-4cf2-b7d9-10bcaab223d5 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:12.413813 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-61d6c625-cf2c-409d-b227-fd45b6f65d88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e03b5fbc07774d43a5344561bbc87ecc'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:12.430142 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-61d6c625-cf2c-409d-b227-fd45b6f65d88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e03b5fbc07774d43a5344561bbc87ecc'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:12.430785 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-61d6c625-cf2c-409d-b227-fd45b6f65d88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e03b5fbc07774d43a5344561bbc87ecc'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:12.439102 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1034/2077] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:15:12 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:15:12.442193 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-61d6c625-cf2c-409d-b227-fd45b6f65d88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e03b5fbc07774d43a5344561bbc87ecc'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:12.443456 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e8a13e-9b67-4cf2-b7d9-10bcaab223d5 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:12.444145 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-61d6c625-cf2c-409d-b227-fd45b6f65d88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e03b5fbc07774d43a5344561bbc87ecc'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:12.449150 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-87e8a13e-9b67-4cf2-b7d9-10bcaab223d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:12.464783 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:12.464932 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e8a13e-9b67-4cf2-b7d9-10bcaab223d5 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:12.471095 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26bf442-e7c3-4494-a15a-5f8b830778b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:12.472216 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26bf442-e7c3-4494-a15a-5f8b830778b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:12.472509 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26bf442-e7c3-4494-a15a-5f8b830778b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:12.473136 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26bf442-e7c3-4494-a15a-5f8b830778b4 admin admin] PATH_INFO: `/v3/projects/81f786893ba84b829555a35286638f0d/users/1ec92d8d8dae4c4d90ba2eef8ce60dab/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:12.477049 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1044/2078] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:15:12 2022] DELETE /identity/v3/users/e03b5fbc07774d43a5344561bbc87ecc => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:12.477219 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26bf442-e7c3-4494-a15a-5f8b830778b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81f786893ba84b829555a35286638f0d, user_id=1ec92d8d8dae4c4d90ba2eef8ce60dab, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:12.479428 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26bf442-e7c3-4494-a15a-5f8b830778b4 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:12.488199 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:12.496970 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1035/2079] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:15:12 2022] PUT /identity/v3/projects/81f786893ba84b829555a35286638f0d/users/1ec92d8d8dae4c4d90ba2eef8ce60dab/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:12.508684 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f506c5-30cb-47a6-a0e6-99aff8957bf7 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:12.509044 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f506c5-30cb-47a6-a0e6-99aff8957bf7 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:12.509506 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f506c5-30cb-47a6-a0e6-99aff8957bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:12.548136 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-a8f506c5-30cb-47a6-a0e6-99aff8957bf7 None None] MFA Rules not processed for user `1ec92d8d8dae4c4d90ba2eef8ce60dab`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:12.549736 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-a8f506c5-30cb-47a6-a0e6-99aff8957bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:12.637431 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1045/2080] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:12 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:15:12.648851 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26bf442-e7c3-4494-a15a-5f8b830778b4 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:12.652750 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-c26bf442-e7c3-4494-a15a-5f8b830778b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:12.720738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26bf442-e7c3-4494-a15a-5f8b830778b4 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:12.725039 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4978c9-5f6e-4182-b82a-a9a5dedbb3ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:12.725748 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4978c9-5f6e-4182-b82a-a9a5dedbb3ba service cinder] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:12.726030 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4978c9-5f6e-4182-b82a-a9a5dedbb3ba service cinder] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:12.726303 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4978c9-5f6e-4182-b82a-a9a5dedbb3ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:12.735899 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4978c9-5f6e-4182-b82a-a9a5dedbb3ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:12.737353 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4978c9-5f6e-4182-b82a-a9a5dedbb3ba service cinder] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:12.753438 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1036/2081] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:15:12 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:15:12.802598 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f506c5-30cb-47a6-a0e6-99aff8957bf7 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:12.806617 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-a8f506c5-30cb-47a6-a0e6-99aff8957bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:12.852094 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4978c9-5f6e-4182-b82a-a9a5dedbb3ba service cinder] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:12.868272 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4978c9-5f6e-4182-b82a-a9a5dedbb3ba service cinder] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:12.872951 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db4a489-1bc6-4092-9f4b-6eb938cd36df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:12.873635 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4a489-1bc6-4092-9f4b-6eb938cd36df service cinder] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:12.873992 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4a489-1bc6-4092-9f4b-6eb938cd36df service cinder] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:12.874736 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4a489-1bc6-4092-9f4b-6eb938cd36df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:12.877318 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-6db4a489-1bc6-4092-9f4b-6eb938cd36df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:12.912169 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f506c5-30cb-47a6-a0e6-99aff8957bf7 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:12.918458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cad01f0-5aaf-4c83-b555-c7a8d83b431e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:12.919457 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cad01f0-5aaf-4c83-b555-c7a8d83b431e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:12.919921 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cad01f0-5aaf-4c83-b555-c7a8d83b431e admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:12.920403 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cad01f0-5aaf-4c83-b555-c7a8d83b431e admin admin] PATH_INFO: `/v3/projects/ff1bfe6f011143e7a460256e28ef43ce` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:12.922173 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cad01f0-5aaf-4c83-b555-c7a8d83b431e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff1bfe6f011143e7a460256e28ef43ce)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:12.924602 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cad01f0-5aaf-4c83-b555-c7a8d83b431e admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:12.953213 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db4a489-1bc6-4092-9f4b-6eb938cd36df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:12.954651 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db4a489-1bc6-4092-9f4b-6eb938cd36df service cinder] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:12.978291 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1037/2082] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:15:12 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 12:15:12.986154 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-7cad01f0-5aaf-4c83-b555-c7a8d83b431e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff1bfe6f011143e7a460256e28ef43ce', 'request_id': 'req-7cad01f0-5aaf-4c83-b555-c7a8d83b431e'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:12.994700 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-7cad01f0-5aaf-4c83-b555-c7a8d83b431e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff1bfe6f011143e7a460256e28ef43ce', 'request_id': 'req-7cad01f0-5aaf-4c83-b555-c7a8d83b431e'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:13.007875 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-7cad01f0-5aaf-4c83-b555-c7a8d83b431e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff1bfe6f011143e7a460256e28ef43ce', 'request_id': 'req-7cad01f0-5aaf-4c83-b555-c7a8d83b431e'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:13.028824 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-7cad01f0-5aaf-4c83-b555-c7a8d83b431e admin admin] The token cache is being invalidate because project ff1bfe6f011143e7a460256e28ef43ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81712) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:15:13.029149 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-7cad01f0-5aaf-4c83-b555-c7a8d83b431e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:13.029799 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-7cad01f0-5aaf-4c83-b555-c7a8d83b431e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:13.031400 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1046/2083] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:15:12 2022] DELETE /identity/v3/projects/ff1bfe6f011143e7a460256e28ef43ce => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:13.036467 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55661f41-a5c1-46ad-9e8e-e54fced97cab None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:13.036819 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55661f41-a5c1-46ad-9e8e-e54fced97cab None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:13.037158 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55661f41-a5c1-46ad-9e8e-e54fced97cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:13.058653 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-55661f41-a5c1-46ad-9e8e-e54fced97cab None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:13.059888 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-55661f41-a5c1-46ad-9e8e-e54fced97cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:13.139972 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1038/2084] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:15:13.148429 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f0e69-e69d-46af-be36-42a4928babd0 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:13.149012 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f0e69-e69d-46af-be36-42a4928babd0 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:13.149468 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f0e69-e69d-46af-be36-42a4928babd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:13.178451 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-d68f0e69-e69d-46af-be36-42a4928babd0 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:13.180358 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-d68f0e69-e69d-46af-be36-42a4928babd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:13.189020 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1047/2085] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:15:13.193269 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55661f41-a5c1-46ad-9e8e-e54fced97cab None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:13.204242 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55661f41-a5c1-46ad-9e8e-e54fced97cab None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:13.210893 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e17ebd6-e5b8-42a8-bd5b-74ff48b6ccf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:13.211730 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e17ebd6-e5b8-42a8-bd5b-74ff48b6ccf4 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:13.212158 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e17ebd6-e5b8-42a8-bd5b-74ff48b6ccf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:13.212678 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e17ebd6-e5b8-42a8-bd5b-74ff48b6ccf4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:13.213439 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e17ebd6-e5b8-42a8-bd5b-74ff48b6ccf4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:15:13.213783 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e17ebd6-e5b8-42a8-bd5b-74ff48b6ccf4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:13.216195 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e17ebd6-e5b8-42a8-bd5b-74ff48b6ccf4 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:13.236253 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1039/2086] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:15:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:15:13.237510 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68f0e69-e69d-46af-be36-42a4928babd0 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:13.254207 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:13.263240 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68f0e69-e69d-46af-be36-42a4928babd0 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:13.276584 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c905ed3a-7013-45fd-9840-d36a52f6d91b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:13.277426 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905ed3a-7013-45fd-9840-d36a52f6d91b admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:13.277784 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905ed3a-7013-45fd-9840-d36a52f6d91b admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:13.278134 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905ed3a-7013-45fd-9840-d36a52f6d91b admin admin] PATH_INFO: `/v3/projects` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:13.279641 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c905ed3a-7013-45fd-9840-d36a52f6d91b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:13.297231 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c905ed3a-7013-45fd-9840-d36a52f6d91b admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:13.340735 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1048/2087] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:15:13 2022] POST /identity/v3/projects => generated 371 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:15:13.341980 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e17ebd6-e5b8-42a8-bd5b-74ff48b6ccf4 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:13.353376 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e17ebd6-e5b8-42a8-bd5b-74ff48b6ccf4 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:13.382968 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:13.419912 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a096dd-51ef-4bf0-b35b-b83e41ee010b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:13.420704 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a096dd-51ef-4bf0-b35b-b83e41ee010b admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:13.421010 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a096dd-51ef-4bf0-b35b-b83e41ee010b admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:13.421284 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a096dd-51ef-4bf0-b35b-b83e41ee010b admin admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:13.421917 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a096dd-51ef-4bf0-b35b-b83e41ee010b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:13.423967 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a096dd-51ef-4bf0-b35b-b83e41ee010b admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:13.447398 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1040/2088] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:15:13 2022] POST /identity/v3/users => generated 286 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:15:13.452294 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c905ed3a-7013-45fd-9840-d36a52f6d91b admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:13.463069 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c905ed3a-7013-45fd-9840-d36a52f6d91b admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:13.467849 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70d7253-995d-4f2f-9075-574cdc4282c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:13.468445 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d7253-995d-4f2f-9075-574cdc4282c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:13.468726 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d7253-995d-4f2f-9075-574cdc4282c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:13.468999 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d7253-995d-4f2f-9075-574cdc4282c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:13.469699 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70d7253-995d-4f2f-9075-574cdc4282c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:13.471236 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70d7253-995d-4f2f-9075-574cdc4282c5 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:13.487937 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1049/2089] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:15:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:15:13.490003 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a096dd-51ef-4bf0-b35b-b83e41ee010b admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:13.505782 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a096dd-51ef-4bf0-b35b-b83e41ee010b admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:13.511587 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99048813-8f79-4789-baa2-334c4e24da53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:13.512202 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99048813-8f79-4789-baa2-334c4e24da53 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:13.512492 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99048813-8f79-4789-baa2-334c4e24da53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:13.512762 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99048813-8f79-4789-baa2-334c4e24da53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:13.513412 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99048813-8f79-4789-baa2-334c4e24da53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:13.515017 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99048813-8f79-4789-baa2-334c4e24da53 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:13.527263 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1041/2090] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:15:13 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:15:13.531622 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70d7253-995d-4f2f-9075-574cdc4282c5 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:13.542879 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70d7253-995d-4f2f-9075-574cdc4282c5 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:13.547892 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c010c770-715c-4366-8b82-b8b495cc3428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:13.548552 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010c770-715c-4366-8b82-b8b495cc3428 admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:13.548833 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010c770-715c-4366-8b82-b8b495cc3428 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:13.549106 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010c770-715c-4366-8b82-b8b495cc3428 admin admin] PATH_INFO: `/v3/projects/ee501ee4ddaf4a82a4e2a36aaaec0ebc/users/ed4cf803a9cc425ab9a5bbc1036c5e44/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:13.568001 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c010c770-715c-4366-8b82-b8b495cc3428 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee501ee4ddaf4a82a4e2a36aaaec0ebc, user_id=ed4cf803a9cc425ab9a5bbc1036c5e44, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:13.570722 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c010c770-715c-4366-8b82-b8b495cc3428 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:13.590747 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1050/2091] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:15:13 2022] PUT /identity/v3/projects/ee501ee4ddaf4a82a4e2a36aaaec0ebc/users/ed4cf803a9cc425ab9a5bbc1036c5e44/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 12:15:13.592667 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d4a01-022e-40b8-a65e-316996241b8b None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:13.593090 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d4a01-022e-40b8-a65e-316996241b8b None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:13.593469 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d4a01-022e-40b8-a65e-316996241b8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:13.621864 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-185d4a01-022e-40b8-a65e-316996241b8b None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:13.623563 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-185d4a01-022e-40b8-a65e-316996241b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:13.699442 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1042/2092] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:15:13.705791 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c010c770-715c-4366-8b82-b8b495cc3428 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:13.707517 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-c010c770-715c-4366-8b82-b8b495cc3428 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:13.769168 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c010c770-715c-4366-8b82-b8b495cc3428 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:13.773023 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6fa7cc-8262-4a0c-a7e8-103632f1a172 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:13.773699 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6fa7cc-8262-4a0c-a7e8-103632f1a172 service neutron] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:13.774086 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6fa7cc-8262-4a0c-a7e8-103632f1a172 service neutron] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:13.774411 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6fa7cc-8262-4a0c-a7e8-103632f1a172 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:13.783460 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6fa7cc-8262-4a0c-a7e8-103632f1a172 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:13.784882 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6fa7cc-8262-4a0c-a7e8-103632f1a172 service neutron] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:13.794143 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1051/2093] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:15:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:15:14.430161 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007f521c-9089-448f-91fd-7ae0388f8ae3 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:14.430646 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007f521c-9089-448f-91fd-7ae0388f8ae3 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:14.430836 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007f521c-9089-448f-91fd-7ae0388f8ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:14.461028 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-007f521c-9089-448f-91fd-7ae0388f8ae3 None None] MFA Rules not processed for user `ed4cf803a9cc425ab9a5bbc1036c5e44`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:14.462199 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-007f521c-9089-448f-91fd-7ae0388f8ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:14.536845 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1043/2094] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:14 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:15:14.538867 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6fa7cc-8262-4a0c-a7e8-103632f1a172 service neutron] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:14.549465 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6fa7cc-8262-4a0c-a7e8-103632f1a172 service neutron] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:14.557902 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad87355e-83c3-4f22-8fc0-4f929520478a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:14.559522 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad87355e-83c3-4f22-8fc0-4f929520478a admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:14.559803 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad87355e-83c3-4f22-8fc0-4f929520478a admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:14.560166 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad87355e-83c3-4f22-8fc0-4f929520478a admin admin] PATH_INFO: `/v3/projects` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:14.560788 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad87355e-83c3-4f22-8fc0-4f929520478a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:14.562705 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad87355e-83c3-4f22-8fc0-4f929520478a admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:14.592375 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1052/2095] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:15:14 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 12:15:14.596584 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007f521c-9089-448f-91fd-7ae0388f8ae3 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:14.608315 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007f521c-9089-448f-91fd-7ae0388f8ae3 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:14.645179 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:14.671900 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e04fd44-d12c-4e28-b205-578b5c1bfe99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:14.672517 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e04fd44-d12c-4e28-b205-578b5c1bfe99 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:14.672803 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e04fd44-d12c-4e28-b205-578b5c1bfe99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:14.673083 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e04fd44-d12c-4e28-b205-578b5c1bfe99 admin admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:14.673711 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e04fd44-d12c-4e28-b205-578b5c1bfe99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:14.675706 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e04fd44-d12c-4e28-b205-578b5c1bfe99 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:14.699561 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1044/2096] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:15:14 2022] POST /identity/v3/users => generated 292 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:15:14.703617 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad87355e-83c3-4f22-8fc0-4f929520478a admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:14.713886 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad87355e-83c3-4f22-8fc0-4f929520478a admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:14.720063 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ed7492-19e2-4818-b191-54b82a8d78b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:14.720664 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed7492-19e2-4818-b191-54b82a8d78b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:14.720942 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed7492-19e2-4818-b191-54b82a8d78b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:14.721215 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed7492-19e2-4818-b191-54b82a8d78b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:14.721851 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ed7492-19e2-4818-b191-54b82a8d78b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:14.723534 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ed7492-19e2-4818-b191-54b82a8d78b5 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:14.736984 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1053/2097] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 12:15:14.742569 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e04fd44-d12c-4e28-b205-578b5c1bfe99 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:14.753393 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e04fd44-d12c-4e28-b205-578b5c1bfe99 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:14.758080 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8bd9ab-3c22-43bb-a369-d3fcce7ab45c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:14.758725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8bd9ab-3c22-43bb-a369-d3fcce7ab45c admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:14.759001 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8bd9ab-3c22-43bb-a369-d3fcce7ab45c admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:14.759299 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8bd9ab-3c22-43bb-a369-d3fcce7ab45c admin admin] PATH_INFO: `/v3/projects/f2955a2f97b14cde82c60fa2d1bf001f/users/7869c1b6616f4e4eb8b226c934bbc8fb/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:14.773929 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8bd9ab-3c22-43bb-a369-d3fcce7ab45c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2955a2f97b14cde82c60fa2d1bf001f, user_id=7869c1b6616f4e4eb8b226c934bbc8fb, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:14.776613 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8bd9ab-3c22-43bb-a369-d3fcce7ab45c admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:14.789966 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1045/2098] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:15:14 2022] PUT /identity/v3/projects/f2955a2f97b14cde82c60fa2d1bf001f/users/7869c1b6616f4e4eb8b226c934bbc8fb/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:14.794072 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ed7492-19e2-4818-b191-54b82a8d78b5 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:14.804680 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ed7492-19e2-4818-b191-54b82a8d78b5 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:14.860072 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a39a54d-2a5a-4cda-935b-a632fdd1045c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:14.860671 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a39a54d-2a5a-4cda-935b-a632fdd1045c admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:14.860948 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a39a54d-2a5a-4cda-935b-a632fdd1045c admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:14.861218 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a39a54d-2a5a-4cda-935b-a632fdd1045c admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:14.861847 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a39a54d-2a5a-4cda-935b-a632fdd1045c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:14.863472 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a39a54d-2a5a-4cda-935b-a632fdd1045c admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:14.873884 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1054/2099] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:15:14.878163 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8bd9ab-3c22-43bb-a369-d3fcce7ab45c admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:14.888802 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8bd9ab-3c22-43bb-a369-d3fcce7ab45c admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:14.893599 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ad9ef7-9306-4856-917b-88109ce1a214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:14.894241 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ad9ef7-9306-4856-917b-88109ce1a214 admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:14.894521 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ad9ef7-9306-4856-917b-88109ce1a214 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:14.894793 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ad9ef7-9306-4856-917b-88109ce1a214 admin admin] PATH_INFO: `/v3/projects/f2955a2f97b14cde82c60fa2d1bf001f/users/7869c1b6616f4e4eb8b226c934bbc8fb/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:14.896768 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ad9ef7-9306-4856-917b-88109ce1a214 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2955a2f97b14cde82c60fa2d1bf001f, user_id=7869c1b6616f4e4eb8b226c934bbc8fb, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:14.899456 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ad9ef7-9306-4856-917b-88109ce1a214 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:14.909339 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1046/2100] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:15:14 2022] PUT /identity/v3/projects/f2955a2f97b14cde82c60fa2d1bf001f/users/7869c1b6616f4e4eb8b226c934bbc8fb/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:15.611505 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff76b1f-8f16-424b-9345-261ec201f6e0 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:15.612047 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff76b1f-8f16-424b-9345-261ec201f6e0 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:15.612047 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff76b1f-8f16-424b-9345-261ec201f6e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:15.651253 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:15.651253 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-0ff76b1f-8f16-424b-9345-261ec201f6e0 None None] MFA Rules not processed for user `7869c1b6616f4e4eb8b226c934bbc8fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:15.651253 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-0ff76b1f-8f16-424b-9345-261ec201f6e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:15.723728 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1055/2101] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:15 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:15:15.735082 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ad9ef7-9306-4856-917b-88109ce1a214 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:15.737005 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-99ad9ef7-9306-4856-917b-88109ce1a214 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:15.802060 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ad9ef7-9306-4856-917b-88109ce1a214 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:15.806570 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3191b352-6506-4052-81b2-c99eae573280 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:15.807176 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3191b352-6506-4052-81b2-c99eae573280 service cinder] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:15.807494 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3191b352-6506-4052-81b2-c99eae573280 service cinder] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:15.807768 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3191b352-6506-4052-81b2-c99eae573280 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:15.864861 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3191b352-6506-4052-81b2-c99eae573280 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:15.867468 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3191b352-6506-4052-81b2-c99eae573280 service cinder] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:15.876642 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1047/2102] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:15:15 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:15:17.284218 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff76b1f-8f16-424b-9345-261ec201f6e0 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:17.285707 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-0ff76b1f-8f16-424b-9345-261ec201f6e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:17.298626 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff76b1f-8f16-424b-9345-261ec201f6e0 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:17.303904 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69955b57-fa1e-4d14-9b4f-1b4609d16dbf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:17.305139 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69955b57-fa1e-4d14-9b4f-1b4609d16dbf service cinder] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:17.305594 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69955b57-fa1e-4d14-9b4f-1b4609d16dbf service cinder] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:17.305897 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69955b57-fa1e-4d14-9b4f-1b4609d16dbf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:17.317110 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69955b57-fa1e-4d14-9b4f-1b4609d16dbf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:17.318676 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69955b57-fa1e-4d14-9b4f-1b4609d16dbf service cinder] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:17.328816 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1056/2103] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:15:17 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:15:19.356894 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3191b352-6506-4052-81b2-c99eae573280 service cinder] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:19.358317 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-3191b352-6506-4052-81b2-c99eae573280 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:19.409723 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:19.428808 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3191b352-6506-4052-81b2-c99eae573280 service cinder] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:19.433307 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ca9448-5133-428d-a9fa-a491541a6b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:19.433930 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca9448-5133-428d-a9fa-a491541a6b61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:19.434204 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca9448-5133-428d-a9fa-a491541a6b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:19.434474 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ca9448-5133-428d-a9fa-a491541a6b61 admin admin] PATH_INFO: `/v3/users/0e77d04ed68e41ae8841f13419b87a84` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:19.435604 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ca9448-5133-428d-a9fa-a491541a6b61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e77d04ed68e41ae8841f13419b87a84)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:19.437505 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ca9448-5133-428d-a9fa-a491541a6b61 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:19.483138 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-c9ca9448-5133-428d-a9fa-a491541a6b61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e77d04ed68e41ae8841f13419b87a84'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:19.489892 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-c9ca9448-5133-428d-a9fa-a491541a6b61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e77d04ed68e41ae8841f13419b87a84'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:19.497443 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-c9ca9448-5133-428d-a9fa-a491541a6b61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e77d04ed68e41ae8841f13419b87a84'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:19.498071 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-c9ca9448-5133-428d-a9fa-a491541a6b61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e77d04ed68e41ae8841f13419b87a84'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:19.510556 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-c9ca9448-5133-428d-a9fa-a491541a6b61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e77d04ed68e41ae8841f13419b87a84'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:19.516216 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1048/2104] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:15:19 2022] DELETE /identity/v3/users/0e77d04ed68e41ae8841f13419b87a84 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:19.520607 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86a6d2-4ae7-459e-a98f-524382da8c08 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:19.521138 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86a6d2-4ae7-459e-a98f-524382da8c08 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:19.521138 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86a6d2-4ae7-459e-a98f-524382da8c08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:19.540169 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-8c86a6d2-4ae7-459e-a98f-524382da8c08 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:19.541377 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-8c86a6d2-4ae7-459e-a98f-524382da8c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:19.606108 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1057/2105] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:15:19.617578 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ca9448-5133-428d-a9fa-a491541a6b61 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:19.618987 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-c9ca9448-5133-428d-a9fa-a491541a6b61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:19.683580 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c86a6d2-4ae7-459e-a98f-524382da8c08 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:19.685201 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-8c86a6d2-4ae7-459e-a98f-524382da8c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:19.686412 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ca9448-5133-428d-a9fa-a491541a6b61 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:19.690111 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb07d7de-0130-4572-852f-bf1a72bff8f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:19.690711 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb07d7de-0130-4572-852f-bf1a72bff8f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:19.690989 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb07d7de-0130-4572-852f-bf1a72bff8f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:19.691293 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb07d7de-0130-4572-852f-bf1a72bff8f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:19.698360 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c86a6d2-4ae7-459e-a98f-524382da8c08 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:19.701112 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb07d7de-0130-4572-852f-bf1a72bff8f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:19.702530 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb07d7de-0130-4572-852f-bf1a72bff8f8 service neutron] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:19.703360 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b08b753-d246-4902-ae9d-1d690193bff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:19.703983 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08b753-d246-4902-ae9d-1d690193bff2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:19.704265 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08b753-d246-4902-ae9d-1d690193bff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:19.704540 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08b753-d246-4902-ae9d-1d690193bff2 admin admin] PATH_INFO: `/v3/users/f2a8f617fed54d649d058fa43aa50314` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:19.705666 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b08b753-d246-4902-ae9d-1d690193bff2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2a8f617fed54d649d058fa43aa50314)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:19.707640 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b08b753-d246-4902-ae9d-1d690193bff2 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:19.713021 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1049/2106] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:15:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:15:19.761325 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-2b08b753-d246-4902-ae9d-1d690193bff2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2a8f617fed54d649d058fa43aa50314'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:19.769552 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-2b08b753-d246-4902-ae9d-1d690193bff2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2a8f617fed54d649d058fa43aa50314'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:19.770226 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-2b08b753-d246-4902-ae9d-1d690193bff2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2a8f617fed54d649d058fa43aa50314'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:19.778919 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-2b08b753-d246-4902-ae9d-1d690193bff2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2a8f617fed54d649d058fa43aa50314'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:19.779836 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-2b08b753-d246-4902-ae9d-1d690193bff2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2a8f617fed54d649d058fa43aa50314'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:19.795746 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:19.801716 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1058/2107] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:15:19 2022] DELETE /identity/v3/users/f2a8f617fed54d649d058fa43aa50314 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 12:15:19.807403 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e7abb7-a46b-4718-a6c2-452593c70046 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:19.807883 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e7abb7-a46b-4718-a6c2-452593c70046 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:19.808312 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e7abb7-a46b-4718-a6c2-452593c70046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:19.828600 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-13e7abb7-a46b-4718-a6c2-452593c70046 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:19.829958 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-13e7abb7-a46b-4718-a6c2-452593c70046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:19.893026 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1050/2108] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:15:19.902893 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b08b753-d246-4902-ae9d-1d690193bff2 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:19.904713 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-2b08b753-d246-4902-ae9d-1d690193bff2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:19.971382 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b08b753-d246-4902-ae9d-1d690193bff2 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:19.979713 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9949fe13-c07d-4189-9fdd-89418357ede8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:19.980789 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949fe13-c07d-4189-9fdd-89418357ede8 service neutron] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:19.981077 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949fe13-c07d-4189-9fdd-89418357ede8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:19.981667 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949fe13-c07d-4189-9fdd-89418357ede8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:19.998556 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9949fe13-c07d-4189-9fdd-89418357ede8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:20.000255 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9949fe13-c07d-4189-9fdd-89418357ede8 service neutron] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:20.010986 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e7abb7-a46b-4718-a6c2-452593c70046 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:20.012891 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-13e7abb7-a46b-4718-a6c2-452593c70046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:20.026017 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e7abb7-a46b-4718-a6c2-452593c70046 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:20.031758 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d40dc64-f001-47e3-8d31-4dc40392742f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:20.032504 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d40dc64-f001-47e3-8d31-4dc40392742f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:20.032919 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d40dc64-f001-47e3-8d31-4dc40392742f admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:20.033710 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d40dc64-f001-47e3-8d31-4dc40392742f admin admin] PATH_INFO: `/v3/projects/2aa27dd0ad0941fea32762d13dca0212` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:20.034336 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1059/2109] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:15:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:15:20.035002 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d40dc64-f001-47e3-8d31-4dc40392742f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2aa27dd0ad0941fea32762d13dca0212)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:20.037165 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d40dc64-f001-47e3-8d31-4dc40392742f admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:20.040116 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:20.122389 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-7d40dc64-f001-47e3-8d31-4dc40392742f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2aa27dd0ad0941fea32762d13dca0212', 'request_id': 'req-7d40dc64-f001-47e3-8d31-4dc40392742f'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:20.131196 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-7d40dc64-f001-47e3-8d31-4dc40392742f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2aa27dd0ad0941fea32762d13dca0212', 'request_id': 'req-7d40dc64-f001-47e3-8d31-4dc40392742f'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:20.141173 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-7d40dc64-f001-47e3-8d31-4dc40392742f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2aa27dd0ad0941fea32762d13dca0212', 'request_id': 'req-7d40dc64-f001-47e3-8d31-4dc40392742f'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:20.160191 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-7d40dc64-f001-47e3-8d31-4dc40392742f admin admin] The token cache is being invalidate because project 2aa27dd0ad0941fea32762d13dca0212 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:15:20.160536 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-7d40dc64-f001-47e3-8d31-4dc40392742f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:20.161223 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-7d40dc64-f001-47e3-8d31-4dc40392742f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:20.165889 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1051/2110] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:15:20 2022] DELETE /identity/v3/projects/2aa27dd0ad0941fea32762d13dca0212 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:20.170753 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9949fe13-c07d-4189-9fdd-89418357ede8 service neutron] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:20.185807 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9949fe13-c07d-4189-9fdd-89418357ede8 service neutron] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:20.279109 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5436c677-c002-4a43-a1bf-d747d5909f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:20.279882 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5436c677-c002-4a43-a1bf-d747d5909f80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:20.280171 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5436c677-c002-4a43-a1bf-d747d5909f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:20.280497 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5436c677-c002-4a43-a1bf-d747d5909f80 admin admin] PATH_INFO: `/v3/users/1ec92d8d8dae4c4d90ba2eef8ce60dab` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:20.281717 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5436c677-c002-4a43-a1bf-d747d5909f80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ec92d8d8dae4c4d90ba2eef8ce60dab)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:20.284810 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5436c677-c002-4a43-a1bf-d747d5909f80 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:20.353608 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-5436c677-c002-4a43-a1bf-d747d5909f80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ec92d8d8dae4c4d90ba2eef8ce60dab'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:20.362429 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-5436c677-c002-4a43-a1bf-d747d5909f80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ec92d8d8dae4c4d90ba2eef8ce60dab'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:20.363246 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-5436c677-c002-4a43-a1bf-d747d5909f80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ec92d8d8dae4c4d90ba2eef8ce60dab'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:20.372724 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-5436c677-c002-4a43-a1bf-d747d5909f80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ec92d8d8dae4c4d90ba2eef8ce60dab'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:20.373509 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-5436c677-c002-4a43-a1bf-d747d5909f80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ec92d8d8dae4c4d90ba2eef8ce60dab'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:20.433070 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d40dc64-f001-47e3-8d31-4dc40392742f admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:20.437215 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-7d40dc64-f001-47e3-8d31-4dc40392742f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:20.442100 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1060/2111] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:15:20 2022] DELETE /identity/v3/users/1ec92d8d8dae4c4d90ba2eef8ce60dab => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:20.473452 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d40dc64-f001-47e3-8d31-4dc40392742f admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:20.556491 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64536ff3-e7fe-4a83-ba51-f121a6f8b7bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:20.557139 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64536ff3-e7fe-4a83-ba51-f121a6f8b7bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:20.557424 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64536ff3-e7fe-4a83-ba51-f121a6f8b7bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:20.557698 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64536ff3-e7fe-4a83-ba51-f121a6f8b7bf admin admin] PATH_INFO: `/v3/projects/59e2b4755fa64f61910a6e4fe5cfb8ed` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:20.559575 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64536ff3-e7fe-4a83-ba51-f121a6f8b7bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=59e2b4755fa64f61910a6e4fe5cfb8ed)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:20.561491 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64536ff3-e7fe-4a83-ba51-f121a6f8b7bf admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:20.628163 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-64536ff3-e7fe-4a83-ba51-f121a6f8b7bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59e2b4755fa64f61910a6e4fe5cfb8ed', 'request_id': 'req-64536ff3-e7fe-4a83-ba51-f121a6f8b7bf'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:20.636753 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-64536ff3-e7fe-4a83-ba51-f121a6f8b7bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59e2b4755fa64f61910a6e4fe5cfb8ed', 'request_id': 'req-64536ff3-e7fe-4a83-ba51-f121a6f8b7bf'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:20.650197 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-64536ff3-e7fe-4a83-ba51-f121a6f8b7bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59e2b4755fa64f61910a6e4fe5cfb8ed', 'request_id': 'req-64536ff3-e7fe-4a83-ba51-f121a6f8b7bf'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:20.664425 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-64536ff3-e7fe-4a83-ba51-f121a6f8b7bf admin admin] The token cache is being invalidate because project 59e2b4755fa64f61910a6e4fe5cfb8ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:15:20.665163 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-64536ff3-e7fe-4a83-ba51-f121a6f8b7bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:20.665845 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-64536ff3-e7fe-4a83-ba51-f121a6f8b7bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:20.671243 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1052/2112] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:15:20 2022] DELETE /identity/v3/projects/59e2b4755fa64f61910a6e4fe5cfb8ed => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:20.671622 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5436c677-c002-4a43-a1bf-d747d5909f80 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:20.673291 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-5436c677-c002-4a43-a1bf-d747d5909f80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:20.688246 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5436c677-c002-4a43-a1bf-d747d5909f80 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:20.714525 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:20.766076 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64536ff3-e7fe-4a83-ba51-f121a6f8b7bf admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:20.768372 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-64536ff3-e7fe-4a83-ba51-f121a6f8b7bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:20.778068 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29c5c0a-d983-4c7d-ae3c-a1d35fb99102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:20.779069 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29c5c0a-d983-4c7d-ae3c-a1d35fb99102 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:20.779625 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29c5c0a-d983-4c7d-ae3c-a1d35fb99102 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:20.780111 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29c5c0a-d983-4c7d-ae3c-a1d35fb99102 admin admin] PATH_INFO: `/v3/users/3cd11892ccf842c4bf24a8821f3005aa` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:20.781904 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29c5c0a-d983-4c7d-ae3c-a1d35fb99102 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cd11892ccf842c4bf24a8821f3005aa)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:20.784411 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29c5c0a-d983-4c7d-ae3c-a1d35fb99102 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:20.803783 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:20.847092 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-a29c5c0a-d983-4c7d-ae3c-a1d35fb99102 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cd11892ccf842c4bf24a8821f3005aa'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:20.855889 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-a29c5c0a-d983-4c7d-ae3c-a1d35fb99102 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cd11892ccf842c4bf24a8821f3005aa'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:20.859648 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-a29c5c0a-d983-4c7d-ae3c-a1d35fb99102 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cd11892ccf842c4bf24a8821f3005aa'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:20.866447 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64536ff3-e7fe-4a83-ba51-f121a6f8b7bf admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:20.868645 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-a29c5c0a-d983-4c7d-ae3c-a1d35fb99102 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cd11892ccf842c4bf24a8821f3005aa'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:20.869364 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-a29c5c0a-d983-4c7d-ae3c-a1d35fb99102 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cd11892ccf842c4bf24a8821f3005aa'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:20.872921 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccfa064-0b75-4685-9ff6-1c4fd0e54c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:20.873848 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccfa064-0b75-4685-9ff6-1c4fd0e54c3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:20.874165 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccfa064-0b75-4685-9ff6-1c4fd0e54c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:20.874538 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccfa064-0b75-4685-9ff6-1c4fd0e54c3e admin admin] PATH_INFO: `/v3/projects/81f786893ba84b829555a35286638f0d` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:20.875743 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccfa064-0b75-4685-9ff6-1c4fd0e54c3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=81f786893ba84b829555a35286638f0d)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:20.877675 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccfa064-0b75-4685-9ff6-1c4fd0e54c3e admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:20.892418 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1061/2113] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:15:20 2022] DELETE /identity/v3/users/3cd11892ccf842c4bf24a8821f3005aa => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 12:15:20.946128 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-4ccfa064-0b75-4685-9ff6-1c4fd0e54c3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81f786893ba84b829555a35286638f0d', 'request_id': 'req-4ccfa064-0b75-4685-9ff6-1c4fd0e54c3e'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:20.959253 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-4ccfa064-0b75-4685-9ff6-1c4fd0e54c3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81f786893ba84b829555a35286638f0d', 'request_id': 'req-4ccfa064-0b75-4685-9ff6-1c4fd0e54c3e'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:20.965644 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-4ccfa064-0b75-4685-9ff6-1c4fd0e54c3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81f786893ba84b829555a35286638f0d', 'request_id': 'req-4ccfa064-0b75-4685-9ff6-1c4fd0e54c3e'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:20.981157 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-4ccfa064-0b75-4685-9ff6-1c4fd0e54c3e admin admin] The token cache is being invalidate because project 81f786893ba84b829555a35286638f0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:15:20.981471 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-4ccfa064-0b75-4685-9ff6-1c4fd0e54c3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:20.982158 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-4ccfa064-0b75-4685-9ff6-1c4fd0e54c3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:20.984077 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1053/2114] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:15:20 2022] DELETE /identity/v3/projects/81f786893ba84b829555a35286638f0d => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:20.988588 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107be9e2-80f1-4882-a40a-99cea15fb218 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:20.988876 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107be9e2-80f1-4882-a40a-99cea15fb218 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:20.989153 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107be9e2-80f1-4882-a40a-99cea15fb218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:21.011403 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-107be9e2-80f1-4882-a40a-99cea15fb218 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:21.012610 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-107be9e2-80f1-4882-a40a-99cea15fb218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:21.079081 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1062/2115] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:15:21.087063 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d71805-10bb-4c23-9b65-7a7667c2056d None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:21.087587 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d71805-10bb-4c23-9b65-7a7667c2056d None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:21.087870 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d71805-10bb-4c23-9b65-7a7667c2056d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:21.109671 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-24d71805-10bb-4c23-9b65-7a7667c2056d None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:21.110981 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-24d71805-10bb-4c23-9b65-7a7667c2056d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:21.120249 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1054/2116] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:15:21.124453 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107be9e2-80f1-4882-a40a-99cea15fb218 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:21.135056 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107be9e2-80f1-4882-a40a-99cea15fb218 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:21.140062 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccae009-c88f-4700-a28d-a4bec022273f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:21.140655 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccae009-c88f-4700-a28d-a4bec022273f admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:21.140937 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccae009-c88f-4700-a28d-a4bec022273f admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:21.141216 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccae009-c88f-4700-a28d-a4bec022273f admin admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:21.141870 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccae009-c88f-4700-a28d-a4bec022273f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:15:21.142179 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccae009-c88f-4700-a28d-a4bec022273f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:21.143824 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccae009-c88f-4700-a28d-a4bec022273f admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:21.158508 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1063/2117] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:15:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:15:21.162825 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d71805-10bb-4c23-9b65-7a7667c2056d None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:21.172728 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d71805-10bb-4c23-9b65-7a7667c2056d None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:21.177953 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a0a681-a7d6-4fcd-a7c5-87f037accd85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:21.179154 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0a681-a7d6-4fcd-a7c5-87f037accd85 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:21.179658 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0a681-a7d6-4fcd-a7c5-87f037accd85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:21.179997 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0a681-a7d6-4fcd-a7c5-87f037accd85 admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:21.180690 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a0a681-a7d6-4fcd-a7c5-87f037accd85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:21.182668 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a0a681-a7d6-4fcd-a7c5-87f037accd85 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:21.216040 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1055/2118] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:15:21 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:15:21.217225 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccae009-c88f-4700-a28d-a4bec022273f admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:21.228407 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccae009-c88f-4700-a28d-a4bec022273f admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:21.257992 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:21.304602 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fae0f9-a6ef-4c33-a687-fc2967858114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:21.305453 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fae0f9-a6ef-4c33-a687-fc2967858114 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:21.305900 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fae0f9-a6ef-4c33-a687-fc2967858114 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:21.306597 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fae0f9-a6ef-4c33-a687-fc2967858114 admin admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:21.307551 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fae0f9-a6ef-4c33-a687-fc2967858114 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:21.310765 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fae0f9-a6ef-4c33-a687-fc2967858114 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:21.343001 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1064/2119] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:15:21 2022] POST /identity/v3/users => generated 290 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 12:15:21.346821 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a0a681-a7d6-4fcd-a7c5-87f037accd85 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:21.363136 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a0a681-a7d6-4fcd-a7c5-87f037accd85 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:21.383012 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e5d71a-29fe-4e67-a849-189cfed9c229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:21.383012 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e5d71a-29fe-4e67-a849-189cfed9c229 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:21.383012 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e5d71a-29fe-4e67-a849-189cfed9c229 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:21.383012 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e5d71a-29fe-4e67-a849-189cfed9c229 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:21.383344 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e5d71a-29fe-4e67-a849-189cfed9c229 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:21.383344 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e5d71a-29fe-4e67-a849-189cfed9c229 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:21.403345 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1056/2120] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:15:21 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:15:21.404633 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fae0f9-a6ef-4c33-a687-fc2967858114 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:21.431962 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fae0f9-a6ef-4c33-a687-fc2967858114 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:21.449322 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583cc8df-9544-4dba-b50c-fb9c7be2d18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:21.459419 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583cc8df-9544-4dba-b50c-fb9c7be2d18e admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:21.459419 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583cc8df-9544-4dba-b50c-fb9c7be2d18e admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:21.459419 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583cc8df-9544-4dba-b50c-fb9c7be2d18e admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:21.459419 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583cc8df-9544-4dba-b50c-fb9c7be2d18e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:21.459419 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583cc8df-9544-4dba-b50c-fb9c7be2d18e admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:21.497632 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1065/2121] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:15:21 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:15:21.498809 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e5d71a-29fe-4e67-a849-189cfed9c229 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:21.509718 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e5d71a-29fe-4e67-a849-189cfed9c229 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:21.525117 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93afe7dd-20a6-4f38-aeb9-1a380b3d35ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:21.525926 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93afe7dd-20a6-4f38-aeb9-1a380b3d35ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:21.526089 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93afe7dd-20a6-4f38-aeb9-1a380b3d35ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:21.526392 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93afe7dd-20a6-4f38-aeb9-1a380b3d35ad admin admin] PATH_INFO: `/v3/projects/d459a7895bed471889858e15681b6202/users/645b47747c834b81959045c893f0967f/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:21.543852 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93afe7dd-20a6-4f38-aeb9-1a380b3d35ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=d459a7895bed471889858e15681b6202, user_id=645b47747c834b81959045c893f0967f, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:21.546453 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93afe7dd-20a6-4f38-aeb9-1a380b3d35ad admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:21.573292 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1057/2122] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:15:21 2022] PUT /identity/v3/projects/d459a7895bed471889858e15681b6202/users/645b47747c834b81959045c893f0967f/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:21.585653 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b04a32-0cc1-475f-b00e-b6c7dbd34eb7 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:21.585990 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b04a32-0cc1-475f-b00e-b6c7dbd34eb7 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:21.586363 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b04a32-0cc1-475f-b00e-b6c7dbd34eb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:21.623860 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-d8b04a32-0cc1-475f-b00e-b6c7dbd34eb7 None None] MFA Rules not processed for user `645b47747c834b81959045c893f0967f`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:21.626289 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-d8b04a32-0cc1-475f-b00e-b6c7dbd34eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:21.769788 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1066/2123] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:15:21.775241 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93afe7dd-20a6-4f38-aeb9-1a380b3d35ad admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:21.808283 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b04a32-0cc1-475f-b00e-b6c7dbd34eb7 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:21.809925 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-d8b04a32-0cc1-475f-b00e-b6c7dbd34eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:21.951402 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:21.983939 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b04a32-0cc1-475f-b00e-b6c7dbd34eb7 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:21.985250 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93afe7dd-20a6-4f38-aeb9-1a380b3d35ad admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:21.989521 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d987b0-30ce-4eb0-8c21-41412965adeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:21.990131 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d987b0-30ce-4eb0-8c21-41412965adeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:21.990481 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d987b0-30ce-4eb0-8c21-41412965adeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:21.990917 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d987b0-30ce-4eb0-8c21-41412965adeb admin admin] PATH_INFO: `/v3/projects/591e422446444d99b36966f117b36694` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:21.991269 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c01949-73c8-4449-95ef-52d8daa7209f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:21.992460 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d987b0-30ce-4eb0-8c21-41412965adeb admin admin] RBAC: Authorizing `identity:delete_project(project_id=591e422446444d99b36966f117b36694)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:21.992596 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c01949-73c8-4449-95ef-52d8daa7209f admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:21.992831 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c01949-73c8-4449-95ef-52d8daa7209f admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:21.993497 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c01949-73c8-4449-95ef-52d8daa7209f admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:21.993727 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c01949-73c8-4449-95ef-52d8daa7209f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:21.994713 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d987b0-30ce-4eb0-8c21-41412965adeb admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:21.995990 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c01949-73c8-4449-95ef-52d8daa7209f admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:22.056506 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1058/2124] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:15:21 2022] POST /identity/v3/projects => generated 377 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:15:22.057945 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c01949-73c8-4449-95ef-52d8daa7209f admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:22.073919 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:22.085334 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-46d987b0-30ce-4eb0-8c21-41412965adeb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '591e422446444d99b36966f117b36694', 'request_id': 'req-46d987b0-30ce-4eb0-8c21-41412965adeb'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:22.099196 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-46d987b0-30ce-4eb0-8c21-41412965adeb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '591e422446444d99b36966f117b36694', 'request_id': 'req-46d987b0-30ce-4eb0-8c21-41412965adeb'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:22.109350 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-46d987b0-30ce-4eb0-8c21-41412965adeb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '591e422446444d99b36966f117b36694', 'request_id': 'req-46d987b0-30ce-4eb0-8c21-41412965adeb'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:22.122088 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-46d987b0-30ce-4eb0-8c21-41412965adeb admin admin] The token cache is being invalidate because project 591e422446444d99b36966f117b36694 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81712) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:15:22.122461 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-46d987b0-30ce-4eb0-8c21-41412965adeb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:22.123465 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-46d987b0-30ce-4eb0-8c21-41412965adeb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:22.125819 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1067/2125] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:15:21 2022] DELETE /identity/v3/projects/591e422446444d99b36966f117b36694 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:22.135889 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d987b0-30ce-4eb0-8c21-41412965adeb admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:22.139303 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-46d987b0-30ce-4eb0-8c21-41412965adeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:22.152373 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c01949-73c8-4449-95ef-52d8daa7209f admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:22.191736 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe52c4da-0522-423e-8256-a9d55c58020a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:22.192339 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe52c4da-0522-423e-8256-a9d55c58020a admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:22.192625 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe52c4da-0522-423e-8256-a9d55c58020a admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:22.192910 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe52c4da-0522-423e-8256-a9d55c58020a admin admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:22.194095 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe52c4da-0522-423e-8256-a9d55c58020a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:22.197965 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe52c4da-0522-423e-8256-a9d55c58020a admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:22.229206 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1059/2126] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:15:22 2022] POST /identity/v3/users => generated 289 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:15:22.231422 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe52c4da-0522-423e-8256-a9d55c58020a admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:22.236010 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-fe52c4da-0522-423e-8256-a9d55c58020a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:22.257491 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe52c4da-0522-423e-8256-a9d55c58020a admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:22.260230 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d987b0-30ce-4eb0-8c21-41412965adeb admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:22.263766 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c9840a-8bf0-485d-8b41-8a2c7454c169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:22.265187 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c9840a-8bf0-485d-8b41-8a2c7454c169 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:22.265315 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be40289e-f6ef-4eb3-9162-2943926b894b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:22.265700 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c9840a-8bf0-485d-8b41-8a2c7454c169 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:22.265957 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40289e-f6ef-4eb3-9162-2943926b894b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:22.266053 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c9840a-8bf0-485d-8b41-8a2c7454c169 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:22.266239 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40289e-f6ef-4eb3-9162-2943926b894b admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:22.266517 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be40289e-f6ef-4eb3-9162-2943926b894b admin admin] PATH_INFO: `/v3/users/55ef532a0076465fad6145d33a85b1b3` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:22.267068 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c9840a-8bf0-485d-8b41-8a2c7454c169 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:22.267990 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be40289e-f6ef-4eb3-9162-2943926b894b admin admin] RBAC: Authorizing `identity:delete_user(user_id=55ef532a0076465fad6145d33a85b1b3)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:22.269005 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c9840a-8bf0-485d-8b41-8a2c7454c169 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:22.269905 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be40289e-f6ef-4eb3-9162-2943926b894b admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:22.282009 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1060/2127] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:15:22 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 12:15:22.285730 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c9840a-8bf0-485d-8b41-8a2c7454c169 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:22.296814 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c9840a-8bf0-485d-8b41-8a2c7454c169 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:22.302487 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0882fa-1b98-4028-a2e2-72304a83a57b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:22.303028 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0882fa-1b98-4028-a2e2-72304a83a57b admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:22.304480 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0882fa-1b98-4028-a2e2-72304a83a57b admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:22.304480 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0882fa-1b98-4028-a2e2-72304a83a57b admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:22.304984 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0882fa-1b98-4028-a2e2-72304a83a57b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:22.306935 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0882fa-1b98-4028-a2e2-72304a83a57b admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:22.321404 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1061/2128] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:15:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 12:15:22.322541 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0882fa-1b98-4028-a2e2-72304a83a57b admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:22.324144 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-be40289e-f6ef-4eb3-9162-2943926b894b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55ef532a0076465fad6145d33a85b1b3'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:22.333170 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-be40289e-f6ef-4eb3-9162-2943926b894b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55ef532a0076465fad6145d33a85b1b3'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:22.333823 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-be40289e-f6ef-4eb3-9162-2943926b894b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55ef532a0076465fad6145d33a85b1b3'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:22.333984 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0882fa-1b98-4028-a2e2-72304a83a57b admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:22.339708 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c156ea-c4cb-4645-83aa-ce33fc988cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:22.340353 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c156ea-c4cb-4645-83aa-ce33fc988cb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:22.340634 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c156ea-c4cb-4645-83aa-ce33fc988cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:22.340921 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c156ea-c4cb-4645-83aa-ce33fc988cb3 admin admin] PATH_INFO: `/v3/projects/bf4f90a221834dca82e4efcc1469c4be/users/34dc2b45c9b942218cc293ce05633a1b/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:22.342027 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-be40289e-f6ef-4eb3-9162-2943926b894b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55ef532a0076465fad6145d33a85b1b3'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:22.342701 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-be40289e-f6ef-4eb3-9162-2943926b894b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55ef532a0076465fad6145d33a85b1b3'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:22.356869 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c156ea-c4cb-4645-83aa-ce33fc988cb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf4f90a221834dca82e4efcc1469c4be, user_id=34dc2b45c9b942218cc293ce05633a1b, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:22.359631 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c156ea-c4cb-4645-83aa-ce33fc988cb3 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:22.362964 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1068/2129] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:15:22 2022] DELETE /identity/v3/users/55ef532a0076465fad6145d33a85b1b3 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:22.376579 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1062/2130] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:15:22 2022] PUT /identity/v3/projects/bf4f90a221834dca82e4efcc1469c4be/users/34dc2b45c9b942218cc293ce05633a1b/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:22.387707 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e14eee-4300-4844-bc29-e787f000308e None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:22.388097 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e14eee-4300-4844-bc29-e787f000308e None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:22.388432 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e14eee-4300-4844-bc29-e787f000308e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:22.418491 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-28e14eee-4300-4844-bc29-e787f000308e None None] MFA Rules not processed for user `34dc2b45c9b942218cc293ce05633a1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:22.419782 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-28e14eee-4300-4844-bc29-e787f000308e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:22.456627 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:22.508208 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1069/2131] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:15:22.513002 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c156ea-c4cb-4645-83aa-ce33fc988cb3 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:22.514723 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-57c156ea-c4cb-4645-83aa-ce33fc988cb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:22.589013 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c156ea-c4cb-4645-83aa-ce33fc988cb3 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:22.594882 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba55f525-c97d-42f1-8a0f-682326f4fc9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:22.595684 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba55f525-c97d-42f1-8a0f-682326f4fc9a admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:22.596071 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba55f525-c97d-42f1-8a0f-682326f4fc9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:22.596448 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba55f525-c97d-42f1-8a0f-682326f4fc9a admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:22.597306 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba55f525-c97d-42f1-8a0f-682326f4fc9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:22.599349 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba55f525-c97d-42f1-8a0f-682326f4fc9a admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:22.625020 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:22.634162 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1063/2132] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:15:22 2022] POST /identity/v3/projects => generated 379 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 12:15:22.637771 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e14eee-4300-4844-bc29-e787f000308e None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:22.649534 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e14eee-4300-4844-bc29-e787f000308e None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:22.697993 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba55f525-c97d-42f1-8a0f-682326f4fc9a admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:22.699936 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-ba55f525-c97d-42f1-8a0f-682326f4fc9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:22.727826 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec20f3a-b696-4555-a568-fc51c46aaf07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:22.728440 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec20f3a-b696-4555-a568-fc51c46aaf07 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:22.728851 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec20f3a-b696-4555-a568-fc51c46aaf07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:22.729044 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec20f3a-b696-4555-a568-fc51c46aaf07 admin admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:22.729681 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec20f3a-b696-4555-a568-fc51c46aaf07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:22.731641 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec20f3a-b696-4555-a568-fc51c46aaf07 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:22.763244 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1070/2133] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:15:22 2022] POST /identity/v3/users => generated 296 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:15:22.767694 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec20f3a-b696-4555-a568-fc51c46aaf07 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:22.781439 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec20f3a-b696-4555-a568-fc51c46aaf07 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:22.789439 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba55f525-c97d-42f1-8a0f-682326f4fc9a admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:22.789768 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd540e7-9d86-4dc9-9e33-b933fb2e07b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:22.790734 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd540e7-9d86-4dc9-9e33-b933fb2e07b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:22.790948 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd540e7-9d86-4dc9-9e33-b933fb2e07b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:22.791365 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd540e7-9d86-4dc9-9e33-b933fb2e07b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:22.792060 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd540e7-9d86-4dc9-9e33-b933fb2e07b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:22.793675 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd540e7-9d86-4dc9-9e33-b933fb2e07b2 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:22.797970 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5355504c-3fc4-4b01-8aa6-4e32ce828ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:22.798692 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355504c-3fc4-4b01-8aa6-4e32ce828ca5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:22.799043 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355504c-3fc4-4b01-8aa6-4e32ce828ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:22.799419 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355504c-3fc4-4b01-8aa6-4e32ce828ca5 admin admin] PATH_INFO: `/v3/projects/daff1723206944e8a25eb146224e9c2b` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:22.800595 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5355504c-3fc4-4b01-8aa6-4e32ce828ca5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=daff1723206944e8a25eb146224e9c2b)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:22.802483 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5355504c-3fc4-4b01-8aa6-4e32ce828ca5 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:22.805143 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1071/2134] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:15:22 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:15:22.810239 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd540e7-9d86-4dc9-9e33-b933fb2e07b2 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:22.820982 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd540e7-9d86-4dc9-9e33-b933fb2e07b2 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:22.825911 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9841156-f36b-4b91-ac24-3103b1a063e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:22.826560 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9841156-f36b-4b91-ac24-3103b1a063e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:22.826845 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9841156-f36b-4b91-ac24-3103b1a063e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:22.827127 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9841156-f36b-4b91-ac24-3103b1a063e0 admin admin] PATH_INFO: `/v3/projects/f6c69846cdd34a8fa3f653e8e4b1ad4c/users/59b0b25111cf4e06a327f425a7867ff7/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:22.846472 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9841156-f36b-4b91-ac24-3103b1a063e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6c69846cdd34a8fa3f653e8e4b1ad4c, user_id=59b0b25111cf4e06a327f425a7867ff7, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:22.849632 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9841156-f36b-4b91-ac24-3103b1a063e0 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:22.855773 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-5355504c-3fc4-4b01-8aa6-4e32ce828ca5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'daff1723206944e8a25eb146224e9c2b', 'request_id': 'req-5355504c-3fc4-4b01-8aa6-4e32ce828ca5'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:22.862958 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-5355504c-3fc4-4b01-8aa6-4e32ce828ca5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'daff1723206944e8a25eb146224e9c2b', 'request_id': 'req-5355504c-3fc4-4b01-8aa6-4e32ce828ca5'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:22.871816 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1072/2135] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:15:22 2022] PUT /identity/v3/projects/f6c69846cdd34a8fa3f653e8e4b1ad4c/users/59b0b25111cf4e06a327f425a7867ff7/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 12:15:22.876655 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-5355504c-3fc4-4b01-8aa6-4e32ce828ca5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'daff1723206944e8a25eb146224e9c2b', 'request_id': 'req-5355504c-3fc4-4b01-8aa6-4e32ce828ca5'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:22.876795 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9841156-f36b-4b91-ac24-3103b1a063e0 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:22.888538 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-5355504c-3fc4-4b01-8aa6-4e32ce828ca5 admin admin] The token cache is being invalidate because project daff1723206944e8a25eb146224e9c2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:15:22.888808 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-5355504c-3fc4-4b01-8aa6-4e32ce828ca5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:22.889435 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-5355504c-3fc4-4b01-8aa6-4e32ce828ca5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:22.891288 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1064/2136] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:15:22 2022] DELETE /identity/v3/projects/daff1723206944e8a25eb146224e9c2b => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 12:15:22.899154 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a20e8-29b7-4db5-9e76-b0dfa5564075 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:22.899520 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a20e8-29b7-4db5-9e76-b0dfa5564075 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:22.899764 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a20e8-29b7-4db5-9e76-b0dfa5564075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:22.924529 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-fc7a20e8-29b7-4db5-9e76-b0dfa5564075 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:22.925885 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-fc7a20e8-29b7-4db5-9e76-b0dfa5564075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:22.990360 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9841156-f36b-4b91-ac24-3103b1a063e0 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:23.000131 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f2342-50ca-40bf-b44d-8563edbe6908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:23.000749 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f2342-50ca-40bf-b44d-8563edbe6908 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:23.001028 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f2342-50ca-40bf-b44d-8563edbe6908 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:23.001357 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f2342-50ca-40bf-b44d-8563edbe6908 admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:23.001966 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48f2342-50ca-40bf-b44d-8563edbe6908 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:23.003916 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48f2342-50ca-40bf-b44d-8563edbe6908 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:23.017985 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1073/2137] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:15:22 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:15:23.022451 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f2342-50ca-40bf-b44d-8563edbe6908 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:23.024355 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-e48f2342-50ca-40bf-b44d-8563edbe6908 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:23.031245 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1065/2138] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:15:23.037167 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853259c2-04e8-4764-80ab-08e99068f09d None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:23.038151 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853259c2-04e8-4764-80ab-08e99068f09d None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:23.038689 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f2342-50ca-40bf-b44d-8563edbe6908 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:23.038858 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853259c2-04e8-4764-80ab-08e99068f09d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:23.044459 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2451e25-c936-45af-b711-3ebef95df8aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:23.045323 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2451e25-c936-45af-b711-3ebef95df8aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:23.045752 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2451e25-c936-45af-b711-3ebef95df8aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:23.046385 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2451e25-c936-45af-b711-3ebef95df8aa admin admin] PATH_INFO: `/v3/projects/f6c69846cdd34a8fa3f653e8e4b1ad4c/users/59b0b25111cf4e06a327f425a7867ff7/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:23.048809 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2451e25-c936-45af-b711-3ebef95df8aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6c69846cdd34a8fa3f653e8e4b1ad4c, user_id=59b0b25111cf4e06a327f425a7867ff7, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:23.052242 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2451e25-c936-45af-b711-3ebef95df8aa admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:23.078558 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1074/2139] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:15:23 2022] PUT /identity/v3/projects/f6c69846cdd34a8fa3f653e8e4b1ad4c/users/59b0b25111cf4e06a327f425a7867ff7/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 12:15:23.082579 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d13744b-a9be-48d0-8e2a-445ee3601eb5 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:23.082880 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d13744b-a9be-48d0-8e2a-445ee3601eb5 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:23.083154 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d13744b-a9be-48d0-8e2a-445ee3601eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:23.089033 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-853259c2-04e8-4764-80ab-08e99068f09d None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:23.090575 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-853259c2-04e8-4764-80ab-08e99068f09d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:23.129719 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:23.140810 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:23.143233 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-4d13744b-a9be-48d0-8e2a-445ee3601eb5 None None] MFA Rules not processed for user `59b0b25111cf4e06a327f425a7867ff7`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:23.145232 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-4d13744b-a9be-48d0-8e2a-445ee3601eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:23.234116 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1066/2140] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:15:23.239458 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853259c2-04e8-4764-80ab-08e99068f09d None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:23.250498 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853259c2-04e8-4764-80ab-08e99068f09d None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:23.257943 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6630751c-04d7-4071-a662-0567f26f3259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:23.258656 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6630751c-04d7-4071-a662-0567f26f3259 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:23.259012 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6630751c-04d7-4071-a662-0567f26f3259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:23.260229 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6630751c-04d7-4071-a662-0567f26f3259 admin admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:23.260897 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6630751c-04d7-4071-a662-0567f26f3259 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:15:23.261470 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6630751c-04d7-4071-a662-0567f26f3259 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:23.263136 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6630751c-04d7-4071-a662-0567f26f3259 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:23.283807 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1067/2141] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:15:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:15:23.290109 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6630751c-04d7-4071-a662-0567f26f3259 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:23.302480 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6630751c-04d7-4071-a662-0567f26f3259 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:23.312322 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1075/2142] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:23 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:15:23.319905 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a78359a-f9ef-446d-974a-39db7e9e3f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:23.320733 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a78359a-f9ef-446d-974a-39db7e9e3f42 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:23.321014 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a78359a-f9ef-446d-974a-39db7e9e3f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:23.321320 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a78359a-f9ef-446d-974a-39db7e9e3f42 admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:23.322001 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a78359a-f9ef-446d-974a-39db7e9e3f42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:23.324417 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a78359a-f9ef-446d-974a-39db7e9e3f42 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:23.327891 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d13744b-a9be-48d0-8e2a-445ee3601eb5 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:23.329472 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-4d13744b-a9be-48d0-8e2a-445ee3601eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:23.360648 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1068/2143] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:15:23 2022] POST /identity/v3/projects => generated 371 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:15:23.361789 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a78359a-f9ef-446d-974a-39db7e9e3f42 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:23.410393 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d13744b-a9be-48d0-8e2a-445ee3601eb5 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:23.444844 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a78359a-f9ef-446d-974a-39db7e9e3f42 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:23.450671 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71386202-6c2a-4868-a6ab-531c0843f99f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:23.451312 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71386202-6c2a-4868-a6ab-531c0843f99f admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:23.451591 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71386202-6c2a-4868-a6ab-531c0843f99f admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:23.451858 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71386202-6c2a-4868-a6ab-531c0843f99f admin admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:23.452474 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71386202-6c2a-4868-a6ab-531c0843f99f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:23.454390 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71386202-6c2a-4868-a6ab-531c0843f99f admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:23.471996 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c997e8f-9f5d-4f16-b3dd-656af4913155 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:23.472605 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c997e8f-9f5d-4f16-b3dd-656af4913155 service cinder] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:23.472881 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c997e8f-9f5d-4f16-b3dd-656af4913155 service cinder] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:23.473152 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c997e8f-9f5d-4f16-b3dd-656af4913155 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:23.480205 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1069/2144] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:15:23 2022] POST /identity/v3/users => generated 286 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 12:15:23.481207 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71386202-6c2a-4868-a6ab-531c0843f99f admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:23.493961 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71386202-6c2a-4868-a6ab-531c0843f99f admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:23.498896 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cee22f8-f586-4771-8e1d-b054af6a36f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:23.499515 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cee22f8-f586-4771-8e1d-b054af6a36f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:23.500248 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cee22f8-f586-4771-8e1d-b054af6a36f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:23.500248 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cee22f8-f586-4771-8e1d-b054af6a36f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:23.500779 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cee22f8-f586-4771-8e1d-b054af6a36f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:23.502370 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cee22f8-f586-4771-8e1d-b054af6a36f1 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:23.513609 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1070/2145] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:15:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:15:23.517598 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cee22f8-f586-4771-8e1d-b054af6a36f1 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:23.527827 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cee22f8-f586-4771-8e1d-b054af6a36f1 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:23.532428 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0356a90-029b-4469-82b5-ae6b139c37bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:23.533007 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0356a90-029b-4469-82b5-ae6b139c37bf admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:23.533287 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0356a90-029b-4469-82b5-ae6b139c37bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:23.533555 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0356a90-029b-4469-82b5-ae6b139c37bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:23.534252 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0356a90-029b-4469-82b5-ae6b139c37bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:23.535784 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0356a90-029b-4469-82b5-ae6b139c37bf admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:23.542399 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c997e8f-9f5d-4f16-b3dd-656af4913155 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:23.543854 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c997e8f-9f5d-4f16-b3dd-656af4913155 service cinder] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:23.549186 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1071/2146] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:15:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:15:23.554543 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0356a90-029b-4469-82b5-ae6b139c37bf admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:23.554908 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1076/2147] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:15:23 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:15:23.566098 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0356a90-029b-4469-82b5-ae6b139c37bf admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:23.575781 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59cbb7e-19a3-475b-a7c9-73ec4685cb43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:23.578549 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59cbb7e-19a3-475b-a7c9-73ec4685cb43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:23.579071 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59cbb7e-19a3-475b-a7c9-73ec4685cb43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:23.579636 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59cbb7e-19a3-475b-a7c9-73ec4685cb43 admin admin] PATH_INFO: `/v3/projects/2f842dc24f364fbda75ed19d2001bd61/users/0ed4572fbda84ba089f65ca75afa6bdd/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:23.606601 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59cbb7e-19a3-475b-a7c9-73ec4685cb43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f842dc24f364fbda75ed19d2001bd61, user_id=0ed4572fbda84ba089f65ca75afa6bdd, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:23.611663 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59cbb7e-19a3-475b-a7c9-73ec4685cb43 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:23.620109 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c997e8f-9f5d-4f16-b3dd-656af4913155 service cinder] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:23.621822 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-8c997e8f-9f5d-4f16-b3dd-656af4913155 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:23.627065 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1072/2148] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:15:23 2022] PUT /identity/v3/projects/2f842dc24f364fbda75ed19d2001bd61/users/0ed4572fbda84ba089f65ca75afa6bdd/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 12:15:23.628089 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:23.634369 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c997e8f-9f5d-4f16-b3dd-656af4913155 service cinder] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:23.638580 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4570dcdb-f114-42ad-8567-0e34514b351d None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:23.638866 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4570dcdb-f114-42ad-8567-0e34514b351d None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:23.639143 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4570dcdb-f114-42ad-8567-0e34514b351d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:23.666055 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-4570dcdb-f114-42ad-8567-0e34514b351d None None] MFA Rules not processed for user `0ed4572fbda84ba089f65ca75afa6bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:23.667300 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-4570dcdb-f114-42ad-8567-0e34514b351d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:23.686164 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df894d7-1854-42bb-9014-929a9f4de8d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:23.686772 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df894d7-1854-42bb-9014-929a9f4de8d6 service cinder] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:23.687049 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df894d7-1854-42bb-9014-929a9f4de8d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:23.687352 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df894d7-1854-42bb-9014-929a9f4de8d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:23.689259 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-2df894d7-1854-42bb-9014-929a9f4de8d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:23.704615 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:23.742600 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1073/2149] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:23 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:15:23.753539 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4570dcdb-f114-42ad-8567-0e34514b351d None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:23.755814 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df894d7-1854-42bb-9014-929a9f4de8d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:23.757809 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df894d7-1854-42bb-9014-929a9f4de8d6 service cinder] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:23.763681 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4570dcdb-f114-42ad-8567-0e34514b351d None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:23.766911 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1077/2150] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:15:23 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 12:15:23.767363 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7295ffcf-0f0b-4ba4-88da-1a55bbcce53b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:23.767964 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7295ffcf-0f0b-4ba4-88da-1a55bbcce53b service cinder] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:23.768242 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7295ffcf-0f0b-4ba4-88da-1a55bbcce53b service cinder] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:23.768514 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7295ffcf-0f0b-4ba4-88da-1a55bbcce53b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:23.778741 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7295ffcf-0f0b-4ba4-88da-1a55bbcce53b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:23.780203 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7295ffcf-0f0b-4ba4-88da-1a55bbcce53b service cinder] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:23.789824 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1074/2151] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:15:23 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 12:15:23.816852 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df894d7-1854-42bb-9014-929a9f4de8d6 service cinder] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:23.886328 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df894d7-1854-42bb-9014-929a9f4de8d6 service cinder] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:23.891207 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b485818c-86e2-407e-af87-71ba448b9ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:23.891868 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485818c-86e2-407e-af87-71ba448b9ed9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:23.892168 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485818c-86e2-407e-af87-71ba448b9ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:23.892441 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485818c-86e2-407e-af87-71ba448b9ed9 admin admin] PATH_INFO: `/v3/users/0ed4572fbda84ba089f65ca75afa6bdd` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:23.893579 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b485818c-86e2-407e-af87-71ba448b9ed9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ed4572fbda84ba089f65ca75afa6bdd)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:23.895612 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b485818c-86e2-407e-af87-71ba448b9ed9 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:23.946000 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-b485818c-86e2-407e-af87-71ba448b9ed9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ed4572fbda84ba089f65ca75afa6bdd'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:23.953367 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-b485818c-86e2-407e-af87-71ba448b9ed9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ed4572fbda84ba089f65ca75afa6bdd'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:23.954214 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-b485818c-86e2-407e-af87-71ba448b9ed9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ed4572fbda84ba089f65ca75afa6bdd'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:23.959308 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:23.962730 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-b485818c-86e2-407e-af87-71ba448b9ed9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ed4572fbda84ba089f65ca75afa6bdd'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:23.964185 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-b485818c-86e2-407e-af87-71ba448b9ed9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ed4572fbda84ba089f65ca75afa6bdd'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:23.987652 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1078/2152] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:15:23 2022] DELETE /identity/v3/users/0ed4572fbda84ba089f65ca75afa6bdd => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:23.993957 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef37c9-4017-492a-9f43-e1a913f57b36 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:23.994274 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef37c9-4017-492a-9f43-e1a913f57b36 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:23.994517 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ef37c9-4017-492a-9f43-e1a913f57b36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:24.018391 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-62ef37c9-4017-492a-9f43-e1a913f57b36 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:24.019644 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-62ef37c9-4017-492a-9f43-e1a913f57b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:24.088510 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1075/2153] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:15:24.106363 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b485818c-86e2-407e-af87-71ba448b9ed9 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:24.107902 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-b485818c-86e2-407e-af87-71ba448b9ed9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:24.186724 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b485818c-86e2-407e-af87-71ba448b9ed9 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:24.202040 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd74cd6-ea8d-4b57-abd4-067fd71e41fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:24.202040 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd74cd6-ea8d-4b57-abd4-067fd71e41fa service neutron] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:24.202040 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd74cd6-ea8d-4b57-abd4-067fd71e41fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:24.202040 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd74cd6-ea8d-4b57-abd4-067fd71e41fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:24.215609 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd74cd6-ea8d-4b57-abd4-067fd71e41fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:24.217053 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd74cd6-ea8d-4b57-abd4-067fd71e41fa service neutron] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:24.229145 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1079/2154] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:15:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:15:24.587732 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ef37c9-4017-492a-9f43-e1a913f57b36 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:24.590482 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-62ef37c9-4017-492a-9f43-e1a913f57b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:24.609593 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ef37c9-4017-492a-9f43-e1a913f57b36 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:24.617289 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43ef411-7ce1-478c-b907-0f744cdf0a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:24.619360 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ef411-7ce1-478c-b907-0f744cdf0a39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:24.620001 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ef411-7ce1-478c-b907-0f744cdf0a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:24.620940 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ef411-7ce1-478c-b907-0f744cdf0a39 admin admin] PATH_INFO: `/v3/projects/2f842dc24f364fbda75ed19d2001bd61` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:24.622847 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43ef411-7ce1-478c-b907-0f744cdf0a39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f842dc24f364fbda75ed19d2001bd61)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:24.626684 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43ef411-7ce1-478c-b907-0f744cdf0a39 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:24.702625 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-e43ef411-7ce1-478c-b907-0f744cdf0a39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f842dc24f364fbda75ed19d2001bd61', 'request_id': 'req-e43ef411-7ce1-478c-b907-0f744cdf0a39'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:24.724510 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-e43ef411-7ce1-478c-b907-0f744cdf0a39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f842dc24f364fbda75ed19d2001bd61', 'request_id': 'req-e43ef411-7ce1-478c-b907-0f744cdf0a39'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:24.745903 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-e43ef411-7ce1-478c-b907-0f744cdf0a39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f842dc24f364fbda75ed19d2001bd61', 'request_id': 'req-e43ef411-7ce1-478c-b907-0f744cdf0a39'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:24.781235 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-e43ef411-7ce1-478c-b907-0f744cdf0a39 admin admin] The token cache is being invalidate because project 2f842dc24f364fbda75ed19d2001bd61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:15:24.781730 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-e43ef411-7ce1-478c-b907-0f744cdf0a39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:24.782747 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-e43ef411-7ce1-478c-b907-0f744cdf0a39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:24.784393 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1076/2155] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:15:24 2022] DELETE /identity/v3/projects/2f842dc24f364fbda75ed19d2001bd61 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:24.789401 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2dbce7-e47c-4bc0-ae7c-f0eff98ffabc None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:24.790441 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2dbce7-e47c-4bc0-ae7c-f0eff98ffabc None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:24.790441 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2dbce7-e47c-4bc0-ae7c-f0eff98ffabc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:24.811739 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-2e2dbce7-e47c-4bc0-ae7c-f0eff98ffabc None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:24.813127 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-2e2dbce7-e47c-4bc0-ae7c-f0eff98ffabc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:24.874994 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1080/2156] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:15:24.884837 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa159642-26ca-43ef-a67a-3429339b6c12 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:24.885131 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa159642-26ca-43ef-a67a-3429339b6c12 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:24.885413 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa159642-26ca-43ef-a67a-3429339b6c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:24.909347 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-fa159642-26ca-43ef-a67a-3429339b6c12 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:24.910619 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-fa159642-26ca-43ef-a67a-3429339b6c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:24.920107 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1077/2157] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:15:24.924224 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2dbce7-e47c-4bc0-ae7c-f0eff98ffabc None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:24.935010 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2dbce7-e47c-4bc0-ae7c-f0eff98ffabc None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:24.939882 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bb8146-3432-43b4-a8d6-0a7bcb245e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:24.940482 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bb8146-3432-43b4-a8d6-0a7bcb245e8b admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:24.940761 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bb8146-3432-43b4-a8d6-0a7bcb245e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:24.941031 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bb8146-3432-43b4-a8d6-0a7bcb245e8b admin admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:24.941698 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bb8146-3432-43b4-a8d6-0a7bcb245e8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:15:24.942010 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bb8146-3432-43b4-a8d6-0a7bcb245e8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:24.943746 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bb8146-3432-43b4-a8d6-0a7bcb245e8b admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:24.965273 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1081/2158] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:15:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:15:24.966428 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa159642-26ca-43ef-a67a-3429339b6c12 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:24.976069 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa159642-26ca-43ef-a67a-3429339b6c12 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:24.981197 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd901391-b0c6-43bf-8837-e7a5de2a9432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:24.981991 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd901391-b0c6-43bf-8837-e7a5de2a9432 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:24.982361 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd901391-b0c6-43bf-8837-e7a5de2a9432 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:24.982793 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd901391-b0c6-43bf-8837-e7a5de2a9432 admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:24.983513 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd901391-b0c6-43bf-8837-e7a5de2a9432 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:24.985741 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd901391-b0c6-43bf-8837-e7a5de2a9432 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:25.015521 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1078/2159] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:15:24 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 12:15:25.020413 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bb8146-3432-43b4-a8d6-0a7bcb245e8b admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:25.032044 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bb8146-3432-43b4-a8d6-0a7bcb245e8b admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:25.092340 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da397b7-5329-4311-ad35-dc3949b1641f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:25.092974 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da397b7-5329-4311-ad35-dc3949b1641f admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:25.093251 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da397b7-5329-4311-ad35-dc3949b1641f admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:25.093524 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da397b7-5329-4311-ad35-dc3949b1641f admin admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:25.094141 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da397b7-5329-4311-ad35-dc3949b1641f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:25.096171 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da397b7-5329-4311-ad35-dc3949b1641f admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:25.108281 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:25.120073 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1082/2160] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:15:25 2022] POST /identity/v3/users => generated 286 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 12:15:25.124189 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd901391-b0c6-43bf-8837-e7a5de2a9432 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:25.134771 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd901391-b0c6-43bf-8837-e7a5de2a9432 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:25.139316 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3be0550-bf1a-4abe-8dd0-4d4f10da1d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:25.139914 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be0550-bf1a-4abe-8dd0-4d4f10da1d98 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:25.140194 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be0550-bf1a-4abe-8dd0-4d4f10da1d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:25.140468 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be0550-bf1a-4abe-8dd0-4d4f10da1d98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:25.141133 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3be0550-bf1a-4abe-8dd0-4d4f10da1d98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:25.142723 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3be0550-bf1a-4abe-8dd0-4d4f10da1d98 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:25.148072 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:25.153280 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1079/2161] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:15:25.160260 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da397b7-5329-4311-ad35-dc3949b1641f admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:25.170703 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da397b7-5329-4311-ad35-dc3949b1641f admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:25.176262 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea5025d-d9d2-4e92-ae7b-513ec7ca9b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:25.176661 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea5025d-d9d2-4e92-ae7b-513ec7ca9b70 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:25.176936 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea5025d-d9d2-4e92-ae7b-513ec7ca9b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:25.177206 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea5025d-d9d2-4e92-ae7b-513ec7ca9b70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:25.177844 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea5025d-d9d2-4e92-ae7b-513ec7ca9b70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:25.180141 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea5025d-d9d2-4e92-ae7b-513ec7ca9b70 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:25.190770 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1083/2162] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:15:25.195008 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3be0550-bf1a-4abe-8dd0-4d4f10da1d98 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:25.205503 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3be0550-bf1a-4abe-8dd0-4d4f10da1d98 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:25.211127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ad129f-c6a9-492d-a292-ca5bc2a12ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:25.211526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ad129f-c6a9-492d-a292-ca5bc2a12ead admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:25.211810 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ad129f-c6a9-492d-a292-ca5bc2a12ead admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:25.212114 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ad129f-c6a9-492d-a292-ca5bc2a12ead admin admin] PATH_INFO: `/v3/projects/6315476c549a4763993b43485ac34917/users/db1e513c022c4b369ae9ce73a2f5f52e/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:25.230933 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ad129f-c6a9-492d-a292-ca5bc2a12ead admin admin] RBAC: Authorizing `identity:create_grant(project_id=6315476c549a4763993b43485ac34917, user_id=db1e513c022c4b369ae9ce73a2f5f52e, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:25.233860 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ad129f-c6a9-492d-a292-ca5bc2a12ead admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:25.250025 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1080/2163] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:15:25 2022] PUT /identity/v3/projects/6315476c549a4763993b43485ac34917/users/db1e513c022c4b369ae9ce73a2f5f52e/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:25.260659 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee30408-b468-47cd-97c2-ed2aee4a2368 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:25.261204 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee30408-b468-47cd-97c2-ed2aee4a2368 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:25.261675 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee30408-b468-47cd-97c2-ed2aee4a2368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:25.296244 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-dee30408-b468-47cd-97c2-ed2aee4a2368 None None] MFA Rules not processed for user `db1e513c022c4b369ae9ce73a2f5f52e`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:25.297301 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-dee30408-b468-47cd-97c2-ed2aee4a2368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:25.382216 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1084/2164] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:25 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:15:25.395689 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ad129f-c6a9-492d-a292-ca5bc2a12ead admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:25.398254 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-d7ad129f-c6a9-492d-a292-ca5bc2a12ead admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:25.466233 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ad129f-c6a9-492d-a292-ca5bc2a12ead admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:25.470538 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6354b0-27ee-4815-80cd-0bf5753b69ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:25.471463 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6354b0-27ee-4815-80cd-0bf5753b69ce service cinder] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:25.472220 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6354b0-27ee-4815-80cd-0bf5753b69ce service cinder] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:25.472504 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6354b0-27ee-4815-80cd-0bf5753b69ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:25.494491 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6354b0-27ee-4815-80cd-0bf5753b69ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:25.497772 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6354b0-27ee-4815-80cd-0bf5753b69ce service cinder] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:25.524495 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1081/2165] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:15:25 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:15:26.776555 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee30408-b468-47cd-97c2-ed2aee4a2368 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:26.778036 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-dee30408-b468-47cd-97c2-ed2aee4a2368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:26.853065 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee30408-b468-47cd-97c2-ed2aee4a2368 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:26.857683 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c29c41b-954f-4804-8c66-44633daf8944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:26.858313 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c29c41b-954f-4804-8c66-44633daf8944 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:26.858592 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c29c41b-954f-4804-8c66-44633daf8944 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:26.858865 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c29c41b-954f-4804-8c66-44633daf8944 admin admin] PATH_INFO: `/v3/users/645b47747c834b81959045c893f0967f` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:26.860024 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c29c41b-954f-4804-8c66-44633daf8944 admin admin] RBAC: Authorizing `identity:delete_user(user_id=645b47747c834b81959045c893f0967f)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:26.862400 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c29c41b-954f-4804-8c66-44633daf8944 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:26.929313 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-2c29c41b-954f-4804-8c66-44633daf8944 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '645b47747c834b81959045c893f0967f'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:26.941817 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-2c29c41b-954f-4804-8c66-44633daf8944 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '645b47747c834b81959045c893f0967f'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:26.942795 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-2c29c41b-954f-4804-8c66-44633daf8944 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '645b47747c834b81959045c893f0967f'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:26.952176 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-2c29c41b-954f-4804-8c66-44633daf8944 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '645b47747c834b81959045c893f0967f'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:26.953182 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-2c29c41b-954f-4804-8c66-44633daf8944 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '645b47747c834b81959045c893f0967f'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:26.973480 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:27.005083 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1085/2166] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:15:26 2022] DELETE /identity/v3/users/645b47747c834b81959045c893f0967f => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:27.006791 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ad5ba5-b1f8-434c-b307-142b371ca4d2 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:27.007739 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ad5ba5-b1f8-434c-b307-142b371ca4d2 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:27.008197 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ad5ba5-b1f8-434c-b307-142b371ca4d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:27.035480 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-95ad5ba5-b1f8-434c-b307-142b371ca4d2 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:27.036539 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-95ad5ba5-b1f8-434c-b307-142b371ca4d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:27.148073 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1082/2167] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:15:27.168281 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c29c41b-954f-4804-8c66-44633daf8944 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:27.170055 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-2c29c41b-954f-4804-8c66-44633daf8944 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:27.259139 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c29c41b-954f-4804-8c66-44633daf8944 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:27.263982 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43875193-b9ec-45e0-83a4-1a26149961fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:27.264754 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43875193-b9ec-45e0-83a4-1a26149961fd service neutron] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:27.265040 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43875193-b9ec-45e0-83a4-1a26149961fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:27.265323 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43875193-b9ec-45e0-83a4-1a26149961fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:27.277197 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43875193-b9ec-45e0-83a4-1a26149961fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:27.278636 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43875193-b9ec-45e0-83a4-1a26149961fd service neutron] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:27.290304 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1086/2168] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:15:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:15:27.770934 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ad5ba5-b1f8-434c-b307-142b371ca4d2 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:27.773995 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-95ad5ba5-b1f8-434c-b307-142b371ca4d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:27.807958 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ad5ba5-b1f8-434c-b307-142b371ca4d2 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:27.825681 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce8e987-0d91-4483-b1ac-20feca83050b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:27.826395 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce8e987-0d91-4483-b1ac-20feca83050b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:27.826681 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce8e987-0d91-4483-b1ac-20feca83050b admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:27.826956 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce8e987-0d91-4483-b1ac-20feca83050b admin admin] PATH_INFO: `/v3/projects/d459a7895bed471889858e15681b6202` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:27.828143 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce8e987-0d91-4483-b1ac-20feca83050b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d459a7895bed471889858e15681b6202)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:27.830336 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce8e987-0d91-4483-b1ac-20feca83050b admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:27.908750 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-0ce8e987-0d91-4483-b1ac-20feca83050b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd459a7895bed471889858e15681b6202', 'request_id': 'req-0ce8e987-0d91-4483-b1ac-20feca83050b'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:27.917891 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-0ce8e987-0d91-4483-b1ac-20feca83050b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd459a7895bed471889858e15681b6202', 'request_id': 'req-0ce8e987-0d91-4483-b1ac-20feca83050b'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:27.929057 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-0ce8e987-0d91-4483-b1ac-20feca83050b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd459a7895bed471889858e15681b6202', 'request_id': 'req-0ce8e987-0d91-4483-b1ac-20feca83050b'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:27.961857 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-0ce8e987-0d91-4483-b1ac-20feca83050b admin admin] The token cache is being invalidate because project d459a7895bed471889858e15681b6202 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:15:27.962192 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-0ce8e987-0d91-4483-b1ac-20feca83050b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:27.963137 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-0ce8e987-0d91-4483-b1ac-20feca83050b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:27.968434 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1083/2169] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:15:27 2022] DELETE /identity/v3/projects/d459a7895bed471889858e15681b6202 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:27.970517 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43875193-b9ec-45e0-83a4-1a26149961fd service neutron] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:27.983476 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43875193-b9ec-45e0-83a4-1a26149961fd service neutron] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:28.058625 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3633930a-ce8a-4bc3-8c55-6b0e7c035ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:28.059455 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3633930a-ce8a-4bc3-8c55-6b0e7c035ba5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:28.059800 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3633930a-ce8a-4bc3-8c55-6b0e7c035ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:28.060134 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3633930a-ce8a-4bc3-8c55-6b0e7c035ba5 admin admin] PATH_INFO: `/v3/users/34dc2b45c9b942218cc293ce05633a1b` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:28.061358 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3633930a-ce8a-4bc3-8c55-6b0e7c035ba5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34dc2b45c9b942218cc293ce05633a1b)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:28.063311 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3633930a-ce8a-4bc3-8c55-6b0e7c035ba5 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:28.119621 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-3633930a-ce8a-4bc3-8c55-6b0e7c035ba5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34dc2b45c9b942218cc293ce05633a1b'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:28.137379 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-3633930a-ce8a-4bc3-8c55-6b0e7c035ba5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34dc2b45c9b942218cc293ce05633a1b'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:28.138043 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-3633930a-ce8a-4bc3-8c55-6b0e7c035ba5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34dc2b45c9b942218cc293ce05633a1b'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:28.146261 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-3633930a-ce8a-4bc3-8c55-6b0e7c035ba5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34dc2b45c9b942218cc293ce05633a1b'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:28.147630 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-3633930a-ce8a-4bc3-8c55-6b0e7c035ba5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34dc2b45c9b942218cc293ce05633a1b'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:28.171806 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1087/2170] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:15:27 2022] DELETE /identity/v3/users/34dc2b45c9b942218cc293ce05633a1b => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:28.485222 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce8e987-0d91-4483-b1ac-20feca83050b admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:28.486827 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-0ce8e987-0d91-4483-b1ac-20feca83050b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:28.495906 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:28.511999 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce8e987-0d91-4483-b1ac-20feca83050b admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:28.587089 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc7020-b486-49b1-bede-b72d45f07b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:28.587791 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc7020-b486-49b1-bede-b72d45f07b38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:28.588040 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc7020-b486-49b1-bede-b72d45f07b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:28.588313 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc7020-b486-49b1-bede-b72d45f07b38 admin admin] PATH_INFO: `/v3/projects/bf4f90a221834dca82e4efcc1469c4be` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:28.589473 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdc7020-b486-49b1-bede-b72d45f07b38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf4f90a221834dca82e4efcc1469c4be)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:28.591606 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdc7020-b486-49b1-bede-b72d45f07b38 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:28.698619 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-ebdc7020-b486-49b1-bede-b72d45f07b38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf4f90a221834dca82e4efcc1469c4be', 'request_id': 'req-ebdc7020-b486-49b1-bede-b72d45f07b38'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:28.705586 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-ebdc7020-b486-49b1-bede-b72d45f07b38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf4f90a221834dca82e4efcc1469c4be', 'request_id': 'req-ebdc7020-b486-49b1-bede-b72d45f07b38'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:28.716021 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-ebdc7020-b486-49b1-bede-b72d45f07b38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf4f90a221834dca82e4efcc1469c4be', 'request_id': 'req-ebdc7020-b486-49b1-bede-b72d45f07b38'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:28.739678 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-ebdc7020-b486-49b1-bede-b72d45f07b38 admin admin] The token cache is being invalidate because project bf4f90a221834dca82e4efcc1469c4be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:15:28.742160 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-ebdc7020-b486-49b1-bede-b72d45f07b38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:28.743190 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-ebdc7020-b486-49b1-bede-b72d45f07b38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:28.745310 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1084/2171] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:15:28 2022] DELETE /identity/v3/projects/bf4f90a221834dca82e4efcc1469c4be => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:28.749571 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3633930a-ce8a-4bc3-8c55-6b0e7c035ba5 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:28.753173 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-3633930a-ce8a-4bc3-8c55-6b0e7c035ba5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:28.768752 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3633930a-ce8a-4bc3-8c55-6b0e7c035ba5 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:28.801699 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:28.916128 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa148a0c-d140-4c08-8a87-4734e72ec6f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:28.917373 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa148a0c-d140-4c08-8a87-4734e72ec6f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:28.917718 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa148a0c-d140-4c08-8a87-4734e72ec6f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:28.918186 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa148a0c-d140-4c08-8a87-4734e72ec6f0 admin admin] PATH_INFO: `/v3/users/59b0b25111cf4e06a327f425a7867ff7` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:28.920329 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa148a0c-d140-4c08-8a87-4734e72ec6f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59b0b25111cf4e06a327f425a7867ff7)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:28.922816 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa148a0c-d140-4c08-8a87-4734e72ec6f0 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:29.099925 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-fa148a0c-d140-4c08-8a87-4734e72ec6f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59b0b25111cf4e06a327f425a7867ff7'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:29.128406 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-fa148a0c-d140-4c08-8a87-4734e72ec6f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59b0b25111cf4e06a327f425a7867ff7'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:29.129209 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-fa148a0c-d140-4c08-8a87-4734e72ec6f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59b0b25111cf4e06a327f425a7867ff7'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:29.164221 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-fa148a0c-d140-4c08-8a87-4734e72ec6f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59b0b25111cf4e06a327f425a7867ff7'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:29.165630 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-fa148a0c-d140-4c08-8a87-4734e72ec6f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59b0b25111cf4e06a327f425a7867ff7'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:29.196555 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1088/2172] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:15:28 2022] DELETE /identity/v3/users/59b0b25111cf4e06a327f425a7867ff7 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:29.885691 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc7020-b486-49b1-bede-b72d45f07b38 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:29.887527 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-ebdc7020-b486-49b1-bede-b72d45f07b38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:29.910191 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc7020-b486-49b1-bede-b72d45f07b38 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:29.995720 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094a2264-b4a2-485b-aeb9-c19c8eae26d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:29.996361 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a2264-b4a2-485b-aeb9-c19c8eae26d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:29.996638 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a2264-b4a2-485b-aeb9-c19c8eae26d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:29.996910 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a2264-b4a2-485b-aeb9-c19c8eae26d8 admin admin] PATH_INFO: `/v3/projects/f6c69846cdd34a8fa3f653e8e4b1ad4c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:29.998294 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094a2264-b4a2-485b-aeb9-c19c8eae26d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6c69846cdd34a8fa3f653e8e4b1ad4c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:30.000224 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094a2264-b4a2-485b-aeb9-c19c8eae26d8 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:30.051599 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:30.068920 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-094a2264-b4a2-485b-aeb9-c19c8eae26d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6c69846cdd34a8fa3f653e8e4b1ad4c', 'request_id': 'req-094a2264-b4a2-485b-aeb9-c19c8eae26d8'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:30.092997 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-094a2264-b4a2-485b-aeb9-c19c8eae26d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6c69846cdd34a8fa3f653e8e4b1ad4c', 'request_id': 'req-094a2264-b4a2-485b-aeb9-c19c8eae26d8'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:30.104085 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-094a2264-b4a2-485b-aeb9-c19c8eae26d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6c69846cdd34a8fa3f653e8e4b1ad4c', 'request_id': 'req-094a2264-b4a2-485b-aeb9-c19c8eae26d8'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:30.117786 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-094a2264-b4a2-485b-aeb9-c19c8eae26d8 admin admin] The token cache is being invalidate because project f6c69846cdd34a8fa3f653e8e4b1ad4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:15:30.119711 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-094a2264-b4a2-485b-aeb9-c19c8eae26d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:30.120438 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-094a2264-b4a2-485b-aeb9-c19c8eae26d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:30.125323 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1085/2173] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:15:29 2022] DELETE /identity/v3/projects/f6c69846cdd34a8fa3f653e8e4b1ad4c => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:30.132263 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee77305b-eb14-4ba0-a659-00af7f7e0b60 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:30.133126 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee77305b-eb14-4ba0-a659-00af7f7e0b60 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:30.133401 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee77305b-eb14-4ba0-a659-00af7f7e0b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:30.156067 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-ee77305b-eb14-4ba0-a659-00af7f7e0b60 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:30.157260 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-ee77305b-eb14-4ba0-a659-00af7f7e0b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:30.227813 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1089/2174] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:15:30.237824 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a05787b-94fe-461e-959e-ad0f2721e9e5 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:30.238359 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a05787b-94fe-461e-959e-ad0f2721e9e5 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:30.238649 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a05787b-94fe-461e-959e-ad0f2721e9e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:30.260531 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-3a05787b-94fe-461e-959e-ad0f2721e9e5 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:30.261705 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-3a05787b-94fe-461e-959e-ad0f2721e9e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:30.270818 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1086/2175] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:15:30.278605 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee77305b-eb14-4ba0-a659-00af7f7e0b60 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:30.290392 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee77305b-eb14-4ba0-a659-00af7f7e0b60 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:30.295166 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dd02f6-31b6-4820-9b4d-dc3149b824b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:30.295799 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dd02f6-31b6-4820-9b4d-dc3149b824b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:30.296080 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dd02f6-31b6-4820-9b4d-dc3149b824b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:30.296355 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dd02f6-31b6-4820-9b4d-dc3149b824b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:30.297007 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dd02f6-31b6-4820-9b4d-dc3149b824b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:15:30.297309 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dd02f6-31b6-4820-9b4d-dc3149b824b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:30.298913 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dd02f6-31b6-4820-9b4d-dc3149b824b0 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:30.315550 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1090/2176] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:15:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:15:30.317554 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a05787b-94fe-461e-959e-ad0f2721e9e5 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:30.328083 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a05787b-94fe-461e-959e-ad0f2721e9e5 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:30.332842 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ace14e-6366-45fc-9bc1-65ca0be2aa24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:30.334079 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ace14e-6366-45fc-9bc1-65ca0be2aa24 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:30.334079 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ace14e-6366-45fc-9bc1-65ca0be2aa24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:30.334079 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ace14e-6366-45fc-9bc1-65ca0be2aa24 admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:30.334715 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ace14e-6366-45fc-9bc1-65ca0be2aa24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:30.337489 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ace14e-6366-45fc-9bc1-65ca0be2aa24 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:30.372998 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1087/2177] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:15:30 2022] POST /identity/v3/projects => generated 367 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:15:30.374370 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dd02f6-31b6-4820-9b4d-dc3149b824b0 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:30.385737 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dd02f6-31b6-4820-9b4d-dc3149b824b0 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:30.407486 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:30.408265 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22fbcec-2cc5-431d-b6ae-76c3a37e115e None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:30.408527 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22fbcec-2cc5-431d-b6ae-76c3a37e115e None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:30.408831 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22fbcec-2cc5-431d-b6ae-76c3a37e115e None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:30.410263 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1088/2178] 10.222.0.61 () {54 vars in 921 bytes} [Fri Aug 26 12:15:30 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 26 12:15:30.440103 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c7a4ee-da94-430a-a9cb-e1f78ec0af4e None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:30.440805 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c7a4ee-da94-430a-a9cb-e1f78ec0af4e None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:30.441118 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c7a4ee-da94-430a-a9cb-e1f78ec0af4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:30.464564 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-10c7a4ee-da94-430a-a9cb-e1f78ec0af4e None None] MFA Rules not processed for user `7346e4a29dfb4a0a9768597704be286c`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:30.465939 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-10c7a4ee-da94-430a-a9cb-e1f78ec0af4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:30.472788 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a546c8a-32ed-4249-a33f-bac45dba5d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:30.479841 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a546c8a-32ed-4249-a33f-bac45dba5d47 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:30.480541 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a546c8a-32ed-4249-a33f-bac45dba5d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:30.481701 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a546c8a-32ed-4249-a33f-bac45dba5d47 admin admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:30.483694 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a546c8a-32ed-4249-a33f-bac45dba5d47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:30.489188 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a546c8a-32ed-4249-a33f-bac45dba5d47 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:30.528063 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1091/2179] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:15:30 2022] POST /identity/v3/users => generated 284 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:15:30.528063 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a546c8a-32ed-4249-a33f-bac45dba5d47 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:30.528063 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-8a546c8a-32ed-4249-a33f-bac45dba5d47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:30.544595 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a546c8a-32ed-4249-a33f-bac45dba5d47 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:30.550663 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88caf0b4-0d45-409d-812f-d518e4689c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:30.551349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88caf0b4-0d45-409d-812f-d518e4689c12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:30.551627 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88caf0b4-0d45-409d-812f-d518e4689c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:30.551896 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88caf0b4-0d45-409d-812f-d518e4689c12 admin admin] PATH_INFO: `/v3/users/ff96b716abcd4565866fc3c24cbe91e5` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:30.553047 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88caf0b4-0d45-409d-812f-d518e4689c12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff96b716abcd4565866fc3c24cbe91e5)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:30.555837 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88caf0b4-0d45-409d-812f-d518e4689c12 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:30.577049 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1089/2180] 10.222.0.61 () {58 vars in 1018 bytes} [Fri Aug 26 12:15:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:15:30.577049 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c7a4ee-da94-430a-a9cb-e1f78ec0af4e None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:30.591503 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c7a4ee-da94-430a-a9cb-e1f78ec0af4e None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:30.600419 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae1dc91-5377-4bcf-a13e-2c0fa8a28005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:30.601280 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1dc91-5377-4bcf-a13e-2c0fa8a28005 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:30.601590 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1dc91-5377-4bcf-a13e-2c0fa8a28005 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:30.601918 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1dc91-5377-4bcf-a13e-2c0fa8a28005 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:30.602643 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae1dc91-5377-4bcf-a13e-2c0fa8a28005 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:30.604593 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae1dc91-5377-4bcf-a13e-2c0fa8a28005 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:30.623248 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1090/2181] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:15:30 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:15:30.624515 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae1dc91-5377-4bcf-a13e-2c0fa8a28005 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:30.627664 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-fae1dc91-5377-4bcf-a13e-2c0fa8a28005 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:30.636160 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-88caf0b4-0d45-409d-812f-d518e4689c12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff96b716abcd4565866fc3c24cbe91e5'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:30.648230 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-88caf0b4-0d45-409d-812f-d518e4689c12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff96b716abcd4565866fc3c24cbe91e5'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:30.648434 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae1dc91-5377-4bcf-a13e-2c0fa8a28005 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:30.649227 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-88caf0b4-0d45-409d-812f-d518e4689c12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff96b716abcd4565866fc3c24cbe91e5'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:30.654192 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5854df88-872c-4470-b306-de7f93120486 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7346e4a29dfb4a0a9768597704be286c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:30.654941 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5854df88-872c-4470-b306-de7f93120486 service placement] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:30.655245 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5854df88-872c-4470-b306-de7f93120486 service placement] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:30.655527 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5854df88-872c-4470-b306-de7f93120486 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:30.657774 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-88caf0b4-0d45-409d-812f-d518e4689c12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff96b716abcd4565866fc3c24cbe91e5'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:30.658626 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-88caf0b4-0d45-409d-812f-d518e4689c12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff96b716abcd4565866fc3c24cbe91e5'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:30.658778 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-5854df88-872c-4470-b306-de7f93120486 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:30.671899 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5854df88-872c-4470-b306-de7f93120486 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:30.673539 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5854df88-872c-4470-b306-de7f93120486 service placement] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:30.684425 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1091/2182] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:15:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:15:30.685060 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5854df88-872c-4470-b306-de7f93120486 service placement] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:30.686450 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-5854df88-872c-4470-b306-de7f93120486 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:30.696000 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1092/2183] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:15:30 2022] DELETE /identity/v3/users/ff96b716abcd4565866fc3c24cbe91e5 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:30.710052 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5854df88-872c-4470-b306-de7f93120486 service placement] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:30.806634 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d10e336-0c5b-4c78-9ac6-f7a426d7e056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:30.807284 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d10e336-0c5b-4c78-9ac6-f7a426d7e056 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:30.807574 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d10e336-0c5b-4c78-9ac6-f7a426d7e056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:30.810877 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d10e336-0c5b-4c78-9ac6-f7a426d7e056 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:30.811475 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d10e336-0c5b-4c78-9ac6-f7a426d7e056 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:30.813111 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d10e336-0c5b-4c78-9ac6-f7a426d7e056 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:30.859500 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1092/2184] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:15:30 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:15:30.863578 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88caf0b4-0d45-409d-812f-d518e4689c12 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:30.894413 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88caf0b4-0d45-409d-812f-d518e4689c12 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:30.920672 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b524b2b3-608f-4e0f-be8a-2bdf93bff126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:30.921352 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b524b2b3-608f-4e0f-be8a-2bdf93bff126 admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:30.921643 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b524b2b3-608f-4e0f-be8a-2bdf93bff126 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:30.921924 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b524b2b3-608f-4e0f-be8a-2bdf93bff126 admin admin] PATH_INFO: `/v3/projects/eb95368758194677a6afb4708c6cf6f7/users/710e9c2a16e3484f8d06021d867f1b97/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:30.943390 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b524b2b3-608f-4e0f-be8a-2bdf93bff126 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb95368758194677a6afb4708c6cf6f7, user_id=710e9c2a16e3484f8d06021d867f1b97, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:30.948495 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b524b2b3-608f-4e0f-be8a-2bdf93bff126 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:30.979832 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1093/2185] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:15:30 2022] PUT /identity/v3/projects/eb95368758194677a6afb4708c6cf6f7/users/710e9c2a16e3484f8d06021d867f1b97/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:30.993070 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bc68bc-c086-471b-83a2-405915008ab5 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:30.993070 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bc68bc-c086-471b-83a2-405915008ab5 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:30.993708 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bc68bc-c086-471b-83a2-405915008ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:31.000744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:31.028191 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-71bc68bc-c086-471b-83a2-405915008ab5 None None] MFA Rules not processed for user `710e9c2a16e3484f8d06021d867f1b97`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:31.029629 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-71bc68bc-c086-471b-83a2-405915008ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:31.082228 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b524b2b3-608f-4e0f-be8a-2bdf93bff126 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:31.084330 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-b524b2b3-608f-4e0f-be8a-2bdf93bff126 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:31.147426 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1093/2186] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:30 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:15:31.159379 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bc68bc-c086-471b-83a2-405915008ab5 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:31.160770 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-71bc68bc-c086-471b-83a2-405915008ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:31.214877 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b524b2b3-608f-4e0f-be8a-2bdf93bff126 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:31.222670 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440254fc-cb15-4d74-b950-df547dfb0871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:31.223601 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440254fc-cb15-4d74-b950-df547dfb0871 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:31.223884 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440254fc-cb15-4d74-b950-df547dfb0871 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:31.224197 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440254fc-cb15-4d74-b950-df547dfb0871 admin admin] PATH_INFO: `/v3/projects/e58ce6250ef341b6bdc369d9475f0670` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:31.226240 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440254fc-cb15-4d74-b950-df547dfb0871 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e58ce6250ef341b6bdc369d9475f0670)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:31.229102 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440254fc-cb15-4d74-b950-df547dfb0871 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:31.231411 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:31.321627 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-440254fc-cb15-4d74-b950-df547dfb0871 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e58ce6250ef341b6bdc369d9475f0670', 'request_id': 'req-440254fc-cb15-4d74-b950-df547dfb0871'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:31.324188 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bc68bc-c086-471b-83a2-405915008ab5 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:31.329953 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-440254fc-cb15-4d74-b950-df547dfb0871 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e58ce6250ef341b6bdc369d9475f0670', 'request_id': 'req-440254fc-cb15-4d74-b950-df547dfb0871'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:31.345546 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-440254fc-cb15-4d74-b950-df547dfb0871 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e58ce6250ef341b6bdc369d9475f0670', 'request_id': 'req-440254fc-cb15-4d74-b950-df547dfb0871'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:31.364614 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-440254fc-cb15-4d74-b950-df547dfb0871 admin admin] The token cache is being invalidate because project e58ce6250ef341b6bdc369d9475f0670 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81712) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:15:31.364913 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-440254fc-cb15-4d74-b950-df547dfb0871 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:31.365639 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-440254fc-cb15-4d74-b950-df547dfb0871 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:31.368268 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1094/2187] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:15:31 2022] DELETE /identity/v3/projects/e58ce6250ef341b6bdc369d9475f0670 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 12:15:31.374165 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1504bbe9-d268-4333-90ca-14be399296e0 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:31.374516 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1504bbe9-d268-4333-90ca-14be399296e0 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:31.374804 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1504bbe9-d268-4333-90ca-14be399296e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:31.398377 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26b34d4-63d5-4df3-aecf-918774f1e34d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152fd7c708114f249b9c960165f0fdfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:31.399417 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26b34d4-63d5-4df3-aecf-918774f1e34d service glance] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:31.399912 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26b34d4-63d5-4df3-aecf-918774f1e34d service glance] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:31.400453 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26b34d4-63d5-4df3-aecf-918774f1e34d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:31.400625 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-1504bbe9-d268-4333-90ca-14be399296e0 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:31.402395 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-1504bbe9-d268-4333-90ca-14be399296e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:31.403075 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-b26b34d4-63d5-4df3-aecf-918774f1e34d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:31.480938 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:31.482867 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1095/2188] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:15:31.493558 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf27ce0-e4d9-4689-a36f-d403363e68a9 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:31.494014 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf27ce0-e4d9-4689-a36f-d403363e68a9 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:31.494437 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf27ce0-e4d9-4689-a36f-d403363e68a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:31.508242 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26b34d4-63d5-4df3-aecf-918774f1e34d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:31.509863 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26b34d4-63d5-4df3-aecf-918774f1e34d service glance] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:31.521572 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1094/2189] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:15:31 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 12:15:31.524795 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-8cf27ce0-e4d9-4689-a36f-d403363e68a9 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:31.526221 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-8cf27ce0-e4d9-4689-a36f-d403363e68a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:31.540234 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1096/2190] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:15:31.546568 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26b34d4-63d5-4df3-aecf-918774f1e34d service glance] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:31.560896 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26b34d4-63d5-4df3-aecf-918774f1e34d service glance] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:31.566376 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4f368b-d38c-4c15-b085-fa377e9ad581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:31.567430 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4f368b-d38c-4c15-b085-fa377e9ad581 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:31.567714 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4f368b-d38c-4c15-b085-fa377e9ad581 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:31.567987 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4f368b-d38c-4c15-b085-fa377e9ad581 admin admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:31.569065 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4f368b-d38c-4c15-b085-fa377e9ad581 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:15:31.569437 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4f368b-d38c-4c15-b085-fa377e9ad581 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:31.577623 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4f368b-d38c-4c15-b085-fa377e9ad581 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:31.594494 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1095/2191] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:15:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:15:31.595737 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf27ce0-e4d9-4689-a36f-d403363e68a9 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:31.607260 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf27ce0-e4d9-4689-a36f-d403363e68a9 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:31.617539 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fa0d7c-cb18-4777-9864-e3d7e3546708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:31.618156 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa0d7c-cb18-4777-9864-e3d7e3546708 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:31.618438 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa0d7c-cb18-4777-9864-e3d7e3546708 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:31.618880 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa0d7c-cb18-4777-9864-e3d7e3546708 admin admin] PATH_INFO: `/v3/projects` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:31.619606 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fa0d7c-cb18-4777-9864-e3d7e3546708 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:31.621603 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fa0d7c-cb18-4777-9864-e3d7e3546708 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:31.660606 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1097/2192] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:15:31 2022] POST /identity/v3/projects => generated 365 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 12:15:31.661943 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4f368b-d38c-4c15-b085-fa377e9ad581 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:31.698777 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4f368b-d38c-4c15-b085-fa377e9ad581 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:31.766118 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bec3b5-08f4-439c-8dc0-f654b338dc21 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:31.766404 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bec3b5-08f4-439c-8dc0-f654b338dc21 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:31.766678 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bec3b5-08f4-439c-8dc0-f654b338dc21 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:31.768739 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1098/2193] 10.222.0.61 () {54 vars in 921 bytes} [Fri Aug 26 12:15:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:15:31.799598 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d1350-b8d9-464c-9c4a-1ed96307727f None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:31.799927 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d1350-b8d9-464c-9c4a-1ed96307727f None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:31.800125 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d1350-b8d9-464c-9c4a-1ed96307727f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:31.805549 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9814ec6a-0317-419a-8c04-53d0eb948382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:31.806189 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9814ec6a-0317-419a-8c04-53d0eb948382 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:31.806455 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9814ec6a-0317-419a-8c04-53d0eb948382 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:31.806725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9814ec6a-0317-419a-8c04-53d0eb948382 admin admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:31.807396 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9814ec6a-0317-419a-8c04-53d0eb948382 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:31.811968 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9814ec6a-0317-419a-8c04-53d0eb948382 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:31.826091 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-767d1350-b8d9-464c-9c4a-1ed96307727f None None] MFA Rules not processed for user `7346e4a29dfb4a0a9768597704be286c`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:31.827632 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-767d1350-b8d9-464c-9c4a-1ed96307727f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:31.861412 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1096/2194] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:15:31 2022] POST /identity/v3/users => generated 283 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 12:15:31.868412 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9814ec6a-0317-419a-8c04-53d0eb948382 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:31.901170 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9814ec6a-0317-419a-8c04-53d0eb948382 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:31.910268 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b435e426-8edf-4a0f-92e3-abf3e8f9cea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:31.911488 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b435e426-8edf-4a0f-92e3-abf3e8f9cea8 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:31.911773 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b435e426-8edf-4a0f-92e3-abf3e8f9cea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:31.912043 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b435e426-8edf-4a0f-92e3-abf3e8f9cea8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:31.913378 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b435e426-8edf-4a0f-92e3-abf3e8f9cea8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:31.915840 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b435e426-8edf-4a0f-92e3-abf3e8f9cea8 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:31.916763 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:31.935331 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1097/2195] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:15:31.935529 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1099/2196] 10.222.0.61 () {58 vars in 1018 bytes} [Fri Aug 26 12:15:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:15:31.941286 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b435e426-8edf-4a0f-92e3-abf3e8f9cea8 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:31.948233 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767d1350-b8d9-464c-9c4a-1ed96307727f None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:31.950039 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-767d1350-b8d9-464c-9c4a-1ed96307727f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:31.954736 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b435e426-8edf-4a0f-92e3-abf3e8f9cea8 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:31.968573 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b4853c-7ef4-4c26-9f14-5c9bc49b72b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:31.969200 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b4853c-7ef4-4c26-9f14-5c9bc49b72b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:31.969478 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b4853c-7ef4-4c26-9f14-5c9bc49b72b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:31.969754 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b4853c-7ef4-4c26-9f14-5c9bc49b72b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:31.970392 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b4853c-7ef4-4c26-9f14-5c9bc49b72b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:31.972078 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b4853c-7ef4-4c26-9f14-5c9bc49b72b1 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:31.973132 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767d1350-b8d9-464c-9c4a-1ed96307727f None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:31.980936 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465869cf-94d9-4373-8a9f-2362a7db94f6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7346e4a29dfb4a0a9768597704be286c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:31.981624 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465869cf-94d9-4373-8a9f-2362a7db94f6 service placement] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:31.981926 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465869cf-94d9-4373-8a9f-2362a7db94f6 service placement] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:31.982183 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465869cf-94d9-4373-8a9f-2362a7db94f6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:31.986516 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1098/2197] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:15:31.990612 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b4853c-7ef4-4c26-9f14-5c9bc49b72b1 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:32.010185 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465869cf-94d9-4373-8a9f-2362a7db94f6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:32.012776 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465869cf-94d9-4373-8a9f-2362a7db94f6 service placement] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:32.015102 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b4853c-7ef4-4c26-9f14-5c9bc49b72b1 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:32.034261 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6194180-e4de-4fd0-9677-55aa50ed6646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:32.035584 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6194180-e4de-4fd0-9677-55aa50ed6646 admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:32.036092 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6194180-e4de-4fd0-9677-55aa50ed6646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:32.036554 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6194180-e4de-4fd0-9677-55aa50ed6646 admin admin] PATH_INFO: `/v3/projects/dcb09d33a2664733b014a9c85deedc85/users/0f320b67b6d247f9b31c2efc64a88d9e/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:32.051190 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1100/2198] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:15:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 12:15:32.098689 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6194180-e4de-4fd0-9677-55aa50ed6646 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcb09d33a2664733b014a9c85deedc85, user_id=0f320b67b6d247f9b31c2efc64a88d9e, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:32.105910 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6194180-e4de-4fd0-9677-55aa50ed6646 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:32.131241 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1099/2199] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:15:31 2022] PUT /identity/v3/projects/dcb09d33a2664733b014a9c85deedc85/users/0f320b67b6d247f9b31c2efc64a88d9e/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:32.142351 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f9045-169f-400b-93f0-447c100f0847 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:32.142735 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f9045-169f-400b-93f0-447c100f0847 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:32.143116 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f9045-169f-400b-93f0-447c100f0847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:32.211425 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-e57f9045-169f-400b-93f0-447c100f0847 None None] MFA Rules not processed for user `0f320b67b6d247f9b31c2efc64a88d9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:32.213242 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-e57f9045-169f-400b-93f0-447c100f0847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:32.374873 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1101/2200] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:15:32.379422 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6194180-e4de-4fd0-9677-55aa50ed6646 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:32.463953 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6194180-e4de-4fd0-9677-55aa50ed6646 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:32.468766 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cc7d65-68dd-47a2-9a33-84bd7a239fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:32.470010 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cc7d65-68dd-47a2-9a33-84bd7a239fb3 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:32.470298 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cc7d65-68dd-47a2-9a33-84bd7a239fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:32.470567 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cc7d65-68dd-47a2-9a33-84bd7a239fb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:32.471468 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cc7d65-68dd-47a2-9a33-84bd7a239fb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:32.473140 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cc7d65-68dd-47a2-9a33-84bd7a239fb3 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:32.501778 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:32.512871 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1100/2201] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:15:32 2022] POST /identity/v3/projects => generated 363 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 12:15:32.513835 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57f9045-169f-400b-93f0-447c100f0847 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:32.526567 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57f9045-169f-400b-93f0-447c100f0847 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:32.619308 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b41456-5cf3-4a7a-afeb-091c6636e4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:32.620567 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b41456-5cf3-4a7a-afeb-091c6636e4ba admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:32.621258 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b41456-5cf3-4a7a-afeb-091c6636e4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:32.621605 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b41456-5cf3-4a7a-afeb-091c6636e4ba admin admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:32.622320 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b41456-5cf3-4a7a-afeb-091c6636e4ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:32.624480 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b41456-5cf3-4a7a-afeb-091c6636e4ba admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:32.653065 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1102/2202] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:15:32 2022] POST /identity/v3/users => generated 288 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 12:15:32.657918 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cc7d65-68dd-47a2-9a33-84bd7a239fb3 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:32.673320 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cc7d65-68dd-47a2-9a33-84bd7a239fb3 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:32.676338 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b41456-5cf3-4a7a-afeb-091c6636e4ba admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:32.679355 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-c3b41456-5cf3-4a7a-afeb-091c6636e4ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:32.680568 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb56b94-82ac-4cdf-90a6-02013d427c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:32.681296 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb56b94-82ac-4cdf-90a6-02013d427c65 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:32.681640 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb56b94-82ac-4cdf-90a6-02013d427c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:32.681979 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb56b94-82ac-4cdf-90a6-02013d427c65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:32.682751 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb56b94-82ac-4cdf-90a6-02013d427c65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:32.684570 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb56b94-82ac-4cdf-90a6-02013d427c65 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:32.692763 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b41456-5cf3-4a7a-afeb-091c6636e4ba admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:32.699049 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68411b9-2136-407c-8237-3b7642d31fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:32.699863 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68411b9-2136-407c-8237-3b7642d31fb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:32.700151 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68411b9-2136-407c-8237-3b7642d31fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:32.700426 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68411b9-2136-407c-8237-3b7642d31fb7 admin admin] PATH_INFO: `/v3/users/ed4cf803a9cc425ab9a5bbc1036c5e44` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:32.701778 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68411b9-2136-407c-8237-3b7642d31fb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed4cf803a9cc425ab9a5bbc1036c5e44)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:32.702247 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1101/2203] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:15:32 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:15:32.706666 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb56b94-82ac-4cdf-90a6-02013d427c65 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:32.707252 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68411b9-2136-407c-8237-3b7642d31fb7 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:32.721559 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb56b94-82ac-4cdf-90a6-02013d427c65 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:32.734517 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a354a6e0-e8b3-4f62-898d-27fa3ff7e4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:32.735826 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a354a6e0-e8b3-4f62-898d-27fa3ff7e4d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:32.736200 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a354a6e0-e8b3-4f62-898d-27fa3ff7e4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:32.736685 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a354a6e0-e8b3-4f62-898d-27fa3ff7e4d6 admin admin] PATH_INFO: `/v3/projects/5f32683bda15462dad208e6ac5ce123b/users/cac4441034ae4f0c82ba8fbd23b3043c/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:32.778173 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a354a6e0-e8b3-4f62-898d-27fa3ff7e4d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f32683bda15462dad208e6ac5ce123b, user_id=cac4441034ae4f0c82ba8fbd23b3043c, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:32.782176 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a354a6e0-e8b3-4f62-898d-27fa3ff7e4d6 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:32.815462 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1102/2204] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:15:32 2022] PUT /identity/v3/projects/5f32683bda15462dad208e6ac5ce123b/users/cac4441034ae4f0c82ba8fbd23b3043c/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 12:15:32.819166 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-c68411b9-2136-407c-8237-3b7642d31fb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed4cf803a9cc425ab9a5bbc1036c5e44'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:32.821029 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a354a6e0-e8b3-4f62-898d-27fa3ff7e4d6 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:32.832101 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-c68411b9-2136-407c-8237-3b7642d31fb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed4cf803a9cc425ab9a5bbc1036c5e44'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:32.832787 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-c68411b9-2136-407c-8237-3b7642d31fb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed4cf803a9cc425ab9a5bbc1036c5e44'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:32.839250 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:32.844601 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-c68411b9-2136-407c-8237-3b7642d31fb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed4cf803a9cc425ab9a5bbc1036c5e44'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:32.846003 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-c68411b9-2136-407c-8237-3b7642d31fb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed4cf803a9cc425ab9a5bbc1036c5e44'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:32.880869 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1103/2205] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:15:32 2022] DELETE /identity/v3/users/ed4cf803a9cc425ab9a5bbc1036c5e44 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:32.913287 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a354a6e0-e8b3-4f62-898d-27fa3ff7e4d6 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:32.986007 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c96b24-91c4-402c-971a-10a0f852afaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:32.986618 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c96b24-91c4-402c-971a-10a0f852afaf admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:32.987426 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c96b24-91c4-402c-971a-10a0f852afaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:32.987717 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c96b24-91c4-402c-971a-10a0f852afaf admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:32.988358 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c96b24-91c4-402c-971a-10a0f852afaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:32.990248 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c96b24-91c4-402c-971a-10a0f852afaf admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:33.003754 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1103/2206] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:15:32 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 12:15:33.007931 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68411b9-2136-407c-8237-3b7642d31fb7 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:33.009652 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-c68411b9-2136-407c-8237-3b7642d31fb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:33.028364 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68411b9-2136-407c-8237-3b7642d31fb7 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:33.037250 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1939c6a-736d-4578-8f7b-36be92eab16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:33.038598 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1939c6a-736d-4578-8f7b-36be92eab16d admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:33.038928 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1939c6a-736d-4578-8f7b-36be92eab16d admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:33.039188 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1939c6a-736d-4578-8f7b-36be92eab16d admin admin] PATH_INFO: `/v3/projects/5f32683bda15462dad208e6ac5ce123b/users/cac4441034ae4f0c82ba8fbd23b3043c/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:33.041575 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1939c6a-736d-4578-8f7b-36be92eab16d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f32683bda15462dad208e6ac5ce123b, user_id=cac4441034ae4f0c82ba8fbd23b3043c, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:33.044825 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1939c6a-736d-4578-8f7b-36be92eab16d admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:33.062577 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1104/2207] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:15:33 2022] PUT /identity/v3/projects/5f32683bda15462dad208e6ac5ce123b/users/cac4441034ae4f0c82ba8fbd23b3043c/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 12:15:33.075512 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc15d18-2290-4612-b300-b3964e0ff577 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:33.076018 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc15d18-2290-4612-b300-b3964e0ff577 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:33.076136 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc15d18-2290-4612-b300-b3964e0ff577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:33.086203 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1939c6a-736d-4578-8f7b-36be92eab16d admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:33.089378 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-c1939c6a-736d-4578-8f7b-36be92eab16d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:33.126465 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-9dc15d18-2290-4612-b300-b3964e0ff577 None None] MFA Rules not processed for user `cac4441034ae4f0c82ba8fbd23b3043c`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:33.129138 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-9dc15d18-2290-4612-b300-b3964e0ff577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:33.213870 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:33.236566 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1104/2208] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:33 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:15:33.249150 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc15d18-2290-4612-b300-b3964e0ff577 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:33.251363 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-9dc15d18-2290-4612-b300-b3964e0ff577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:33.254109 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1939c6a-736d-4578-8f7b-36be92eab16d admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:33.263492 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f64fe0-272b-460d-8bdb-77439d83c9b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:33.264510 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f64fe0-272b-460d-8bdb-77439d83c9b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:33.264719 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f64fe0-272b-460d-8bdb-77439d83c9b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:33.265037 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f64fe0-272b-460d-8bdb-77439d83c9b0 admin admin] PATH_INFO: `/v3/projects/ee501ee4ddaf4a82a4e2a36aaaec0ebc` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:33.266263 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f64fe0-272b-460d-8bdb-77439d83c9b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee501ee4ddaf4a82a4e2a36aaaec0ebc)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:33.268502 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f64fe0-272b-460d-8bdb-77439d83c9b0 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:33.355413 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-57f64fe0-272b-460d-8bdb-77439d83c9b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee501ee4ddaf4a82a4e2a36aaaec0ebc', 'request_id': 'req-57f64fe0-272b-460d-8bdb-77439d83c9b0'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:33.355519 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc15d18-2290-4612-b300-b3964e0ff577 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:33.355609 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-57f64fe0-272b-460d-8bdb-77439d83c9b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee501ee4ddaf4a82a4e2a36aaaec0ebc', 'request_id': 'req-57f64fe0-272b-460d-8bdb-77439d83c9b0'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:33.368461 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-57f64fe0-272b-460d-8bdb-77439d83c9b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee501ee4ddaf4a82a4e2a36aaaec0ebc', 'request_id': 'req-57f64fe0-272b-460d-8bdb-77439d83c9b0'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:33.383969 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-57f64fe0-272b-460d-8bdb-77439d83c9b0 admin admin] The token cache is being invalidate because project ee501ee4ddaf4a82a4e2a36aaaec0ebc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81712) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:15:33.384572 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-57f64fe0-272b-460d-8bdb-77439d83c9b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:33.385651 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-57f64fe0-272b-460d-8bdb-77439d83c9b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:33.388360 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1105/2209] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:15:33 2022] DELETE /identity/v3/projects/ee501ee4ddaf4a82a4e2a36aaaec0ebc => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 12:15:33.393654 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f64fe0-272b-460d-8bdb-77439d83c9b0 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:33.396488 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-57f64fe0-272b-460d-8bdb-77439d83c9b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:33.437558 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81f35b6-25fc-4147-94b4-99bceb57c395 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:33.438479 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f35b6-25fc-4147-94b4-99bceb57c395 service cinder] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:33.439789 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f35b6-25fc-4147-94b4-99bceb57c395 service cinder] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:33.440072 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f35b6-25fc-4147-94b4-99bceb57c395 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:33.442570 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-b81f35b6-25fc-4147-94b4-99bceb57c395 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:33.497739 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:33.519424 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f64fe0-272b-460d-8bdb-77439d83c9b0 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:33.523925 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3a38ae-2fc2-4037-bb3a-42c44cc8fac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:33.523925 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a38ae-2fc2-4037-bb3a-42c44cc8fac9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:33.524723 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a38ae-2fc2-4037-bb3a-42c44cc8fac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:33.525727 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a38ae-2fc2-4037-bb3a-42c44cc8fac9 admin admin] PATH_INFO: `/v3/users/7869c1b6616f4e4eb8b226c934bbc8fb` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:33.527636 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3a38ae-2fc2-4037-bb3a-42c44cc8fac9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7869c1b6616f4e4eb8b226c934bbc8fb)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:33.530131 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3a38ae-2fc2-4037-bb3a-42c44cc8fac9 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:33.548711 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81f35b6-25fc-4147-94b4-99bceb57c395 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:33.550150 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81f35b6-25fc-4147-94b4-99bceb57c395 service cinder] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:33.561354 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1105/2210] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:15:33 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 12:15:33.581204 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-3e3a38ae-2fc2-4037-bb3a-42c44cc8fac9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7869c1b6616f4e4eb8b226c934bbc8fb'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:33.590883 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-3e3a38ae-2fc2-4037-bb3a-42c44cc8fac9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7869c1b6616f4e4eb8b226c934bbc8fb'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:33.591647 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-3e3a38ae-2fc2-4037-bb3a-42c44cc8fac9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7869c1b6616f4e4eb8b226c934bbc8fb'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:33.599166 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-3e3a38ae-2fc2-4037-bb3a-42c44cc8fac9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7869c1b6616f4e4eb8b226c934bbc8fb'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:33.599941 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-3e3a38ae-2fc2-4037-bb3a-42c44cc8fac9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7869c1b6616f4e4eb8b226c934bbc8fb'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:33.618110 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1106/2211] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:15:33 2022] DELETE /identity/v3/users/7869c1b6616f4e4eb8b226c934bbc8fb => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:33.830901 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81f35b6-25fc-4147-94b4-99bceb57c395 service cinder] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:33.857622 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81f35b6-25fc-4147-94b4-99bceb57c395 service cinder] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:33.919773 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:33.929267 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212d9fbf-11af-41e8-bb0a-9d474b3872e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:33.929906 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212d9fbf-11af-41e8-bb0a-9d474b3872e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:33.930183 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212d9fbf-11af-41e8-bb0a-9d474b3872e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:33.930448 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212d9fbf-11af-41e8-bb0a-9d474b3872e3 admin admin] PATH_INFO: `/v3/projects/f2955a2f97b14cde82c60fa2d1bf001f` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:33.931615 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212d9fbf-11af-41e8-bb0a-9d474b3872e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2955a2f97b14cde82c60fa2d1bf001f)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:33.933636 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212d9fbf-11af-41e8-bb0a-9d474b3872e3 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:33.983657 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-212d9fbf-11af-41e8-bb0a-9d474b3872e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2955a2f97b14cde82c60fa2d1bf001f', 'request_id': 'req-212d9fbf-11af-41e8-bb0a-9d474b3872e3'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:33.992301 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-212d9fbf-11af-41e8-bb0a-9d474b3872e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2955a2f97b14cde82c60fa2d1bf001f', 'request_id': 'req-212d9fbf-11af-41e8-bb0a-9d474b3872e3'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:34.003084 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-212d9fbf-11af-41e8-bb0a-9d474b3872e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2955a2f97b14cde82c60fa2d1bf001f', 'request_id': 'req-212d9fbf-11af-41e8-bb0a-9d474b3872e3'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:34.014944 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-212d9fbf-11af-41e8-bb0a-9d474b3872e3 admin admin] The token cache is being invalidate because project f2955a2f97b14cde82c60fa2d1bf001f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:15:34.015291 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-212d9fbf-11af-41e8-bb0a-9d474b3872e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:34.015956 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-212d9fbf-11af-41e8-bb0a-9d474b3872e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:34.017444 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1106/2212] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:15:33 2022] DELETE /identity/v3/projects/f2955a2f97b14cde82c60fa2d1bf001f => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:34.023873 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9705cd9-2735-4e2d-b77c-97cec74604bf None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:34.024160 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9705cd9-2735-4e2d-b77c-97cec74604bf None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:34.024436 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9705cd9-2735-4e2d-b77c-97cec74604bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:34.028132 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212d9fbf-11af-41e8-bb0a-9d474b3872e3 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:34.029488 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-212d9fbf-11af-41e8-bb0a-9d474b3872e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:34.044845 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-e9705cd9-2735-4e2d-b77c-97cec74604bf None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:34.046020 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-e9705cd9-2735-4e2d-b77c-97cec74604bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:34.107265 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212d9fbf-11af-41e8-bb0a-9d474b3872e3 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:34.112513 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb7dcb-e07d-47af-9329-4bf3ab3754d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:34.113141 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb7dcb-e07d-47af-9329-4bf3ab3754d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:34.113448 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb7dcb-e07d-47af-9329-4bf3ab3754d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:34.113719 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb7dcb-e07d-47af-9329-4bf3ab3754d0 admin admin] PATH_INFO: `/v3/users/0f320b67b6d247f9b31c2efc64a88d9e` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:34.114845 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfb7dcb-e07d-47af-9329-4bf3ab3754d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f320b67b6d247f9b31c2efc64a88d9e)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:34.116853 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1107/2213] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:15:34.119793 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfb7dcb-e07d-47af-9329-4bf3ab3754d0 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:34.124564 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f09833b-2e58-448a-aec5-505ae7b6c818 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:34.124853 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f09833b-2e58-448a-aec5-505ae7b6c818 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:34.125187 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f09833b-2e58-448a-aec5-505ae7b6c818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:34.145015 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-4f09833b-2e58-448a-aec5-505ae7b6c818 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:34.146200 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-4f09833b-2e58-448a-aec5-505ae7b6c818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:34.156968 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1108/2214] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:15:34.161158 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f09833b-2e58-448a-aec5-505ae7b6c818 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:34.172259 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f09833b-2e58-448a-aec5-505ae7b6c818 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:34.174608 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-edfb7dcb-e07d-47af-9329-4bf3ab3754d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f320b67b6d247f9b31c2efc64a88d9e'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:34.177210 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee69ee91-0831-4c3b-ad6f-5fae682ceced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:34.177815 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee69ee91-0831-4c3b-ad6f-5fae682ceced admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:34.178283 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee69ee91-0831-4c3b-ad6f-5fae682ceced admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:34.178671 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee69ee91-0831-4c3b-ad6f-5fae682ceced admin admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:34.179599 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee69ee91-0831-4c3b-ad6f-5fae682ceced admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:15:34.180129 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee69ee91-0831-4c3b-ad6f-5fae682ceced admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:34.182286 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee69ee91-0831-4c3b-ad6f-5fae682ceced admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:34.183564 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-edfb7dcb-e07d-47af-9329-4bf3ab3754d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f320b67b6d247f9b31c2efc64a88d9e'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:34.191074 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-edfb7dcb-e07d-47af-9329-4bf3ab3754d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f320b67b6d247f9b31c2efc64a88d9e'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:34.191760 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-edfb7dcb-e07d-47af-9329-4bf3ab3754d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f320b67b6d247f9b31c2efc64a88d9e'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:34.195602 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1109/2215] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:15:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:15:34.200398 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee69ee91-0831-4c3b-ad6f-5fae682ceced admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:34.204158 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-edfb7dcb-e07d-47af-9329-4bf3ab3754d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f320b67b6d247f9b31c2efc64a88d9e'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:34.210726 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1107/2216] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:15:34 2022] DELETE /identity/v3/users/0f320b67b6d247f9b31c2efc64a88d9e => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 12:15:34.210905 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee69ee91-0831-4c3b-ad6f-5fae682ceced admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:34.217209 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c8099e-0602-4811-ab04-c56ba3b15431 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:34.217543 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c8099e-0602-4811-ab04-c56ba3b15431 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:34.217822 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c8099e-0602-4811-ab04-c56ba3b15431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:34.233618 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:34.241478 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-e4c8099e-0602-4811-ab04-c56ba3b15431 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:34.244328 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-e4c8099e-0602-4811-ab04-c56ba3b15431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:34.265151 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:34.276292 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e71a0-e113-4664-86ef-aaf446e5b710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:34.276928 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e71a0-e113-4664-86ef-aaf446e5b710 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:34.277213 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e71a0-e113-4664-86ef-aaf446e5b710 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:34.277481 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e71a0-e113-4664-86ef-aaf446e5b710 admin admin] PATH_INFO: `/v3/projects` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:34.278150 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836e71a0-e113-4664-86ef-aaf446e5b710 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:34.280379 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836e71a0-e113-4664-86ef-aaf446e5b710 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:34.308871 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1110/2217] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:15:34 2022] POST /identity/v3/projects => generated 373 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 12:15:34.312927 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e71a0-e113-4664-86ef-aaf446e5b710 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:34.314081 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1108/2218] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:15:34.315740 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-836e71a0-e113-4664-86ef-aaf446e5b710 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:34.324677 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c8099e-0602-4811-ab04-c56ba3b15431 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:34.326100 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-e4c8099e-0602-4811-ab04-c56ba3b15431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:34.385756 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c8099e-0602-4811-ab04-c56ba3b15431 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:34.397543 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e71a0-e113-4664-86ef-aaf446e5b710 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:34.402969 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaab8a7d-7118-4086-8b56-00e17e3b1417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:34.403592 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab8a7d-7118-4086-8b56-00e17e3b1417 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:34.403866 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab8a7d-7118-4086-8b56-00e17e3b1417 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:34.404132 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab8a7d-7118-4086-8b56-00e17e3b1417 admin admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:34.404745 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaab8a7d-7118-4086-8b56-00e17e3b1417 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:34.406663 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaab8a7d-7118-4086-8b56-00e17e3b1417 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:34.429490 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1111/2219] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:15:34 2022] POST /identity/v3/users => generated 287 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 12:15:34.433566 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaab8a7d-7118-4086-8b56-00e17e3b1417 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:34.436515 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f982c6d9-374b-4a3e-9341-c95f5491167d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:34.437118 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f982c6d9-374b-4a3e-9341-c95f5491167d service neutron] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:34.437422 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f982c6d9-374b-4a3e-9341-c95f5491167d service neutron] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:34.437688 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f982c6d9-374b-4a3e-9341-c95f5491167d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:34.443328 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaab8a7d-7118-4086-8b56-00e17e3b1417 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:34.447593 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f982c6d9-374b-4a3e-9341-c95f5491167d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:34.449050 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f982c6d9-374b-4a3e-9341-c95f5491167d service neutron] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:34.449657 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242c4a98-64a7-4a2e-b149-d0253e350044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:34.450419 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242c4a98-64a7-4a2e-b149-d0253e350044 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:34.450705 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242c4a98-64a7-4a2e-b149-d0253e350044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:34.451126 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242c4a98-64a7-4a2e-b149-d0253e350044 admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:34.451940 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242c4a98-64a7-4a2e-b149-d0253e350044 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:34.453750 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242c4a98-64a7-4a2e-b149-d0253e350044 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:34.460393 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1109/2220] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:15:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 12:15:34.471019 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1112/2221] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 12:15:34.475799 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f982c6d9-374b-4a3e-9341-c95f5491167d service neutron] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:34.487610 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f982c6d9-374b-4a3e-9341-c95f5491167d service neutron] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:34.494076 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd614a6-f1f4-4922-8157-f95db3722751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:34.494327 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd614a6-f1f4-4922-8157-f95db3722751 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:34.494603 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd614a6-f1f4-4922-8157-f95db3722751 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:34.494872 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd614a6-f1f4-4922-8157-f95db3722751 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:34.495537 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd614a6-f1f4-4922-8157-f95db3722751 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:34.497343 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd614a6-f1f4-4922-8157-f95db3722751 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:34.510349 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1110/2222] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:15:34.514398 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242c4a98-64a7-4a2e-b149-d0253e350044 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:34.528755 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242c4a98-64a7-4a2e-b149-d0253e350044 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:34.541249 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26940856-10ba-4511-b365-d0ed306faa68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:34.542329 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26940856-10ba-4511-b365-d0ed306faa68 admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:34.542730 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26940856-10ba-4511-b365-d0ed306faa68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:34.543534 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26940856-10ba-4511-b365-d0ed306faa68 admin admin] PATH_INFO: `/v3/projects/85b9d4e830684dca9c0bf1e013b5ca72/users/116651577f2e49e2a4fc6c5ad339d358/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:34.571305 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26940856-10ba-4511-b365-d0ed306faa68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85b9d4e830684dca9c0bf1e013b5ca72, user_id=116651577f2e49e2a4fc6c5ad339d358, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:34.574628 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26940856-10ba-4511-b365-d0ed306faa68 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:34.602933 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1113/2223] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:15:34 2022] PUT /identity/v3/projects/85b9d4e830684dca9c0bf1e013b5ca72/users/116651577f2e49e2a4fc6c5ad339d358/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 12:15:34.614546 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa78c1-2703-43e9-ad16-3a60fd360d5c None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:34.614843 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa78c1-2703-43e9-ad16-3a60fd360d5c None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:34.615309 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa78c1-2703-43e9-ad16-3a60fd360d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:34.658507 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-acaa78c1-2703-43e9-ad16-3a60fd360d5c None None] MFA Rules not processed for user `116651577f2e49e2a4fc6c5ad339d358`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:34.659823 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-acaa78c1-2703-43e9-ad16-3a60fd360d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:34.734360 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:34.767397 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1111/2224] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:15:34.843754 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26940856-10ba-4511-b365-d0ed306faa68 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:34.845475 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-26940856-10ba-4511-b365-d0ed306faa68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:34.932056 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26940856-10ba-4511-b365-d0ed306faa68 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:34.936899 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1b805d-3fe4-4002-9221-713620028d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:34.937532 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1b805d-3fe4-4002-9221-713620028d3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:34.937828 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1b805d-3fe4-4002-9221-713620028d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:34.938097 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1b805d-3fe4-4002-9221-713620028d3d admin admin] PATH_INFO: `/v3/projects/dcb09d33a2664733b014a9c85deedc85` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:34.939256 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1b805d-3fe4-4002-9221-713620028d3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcb09d33a2664733b014a9c85deedc85)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:34.941223 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1b805d-3fe4-4002-9221-713620028d3d admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:34.984899 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:34.997230 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-0b1b805d-3fe4-4002-9221-713620028d3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcb09d33a2664733b014a9c85deedc85', 'request_id': 'req-0b1b805d-3fe4-4002-9221-713620028d3d'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:35.004167 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-0b1b805d-3fe4-4002-9221-713620028d3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcb09d33a2664733b014a9c85deedc85', 'request_id': 'req-0b1b805d-3fe4-4002-9221-713620028d3d'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:35.011445 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-0b1b805d-3fe4-4002-9221-713620028d3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcb09d33a2664733b014a9c85deedc85', 'request_id': 'req-0b1b805d-3fe4-4002-9221-713620028d3d'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:35.021728 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-0b1b805d-3fe4-4002-9221-713620028d3d admin admin] The token cache is being invalidate because project dcb09d33a2664733b014a9c85deedc85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81712) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:15:35.022015 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-0b1b805d-3fe4-4002-9221-713620028d3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:35.022633 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-0b1b805d-3fe4-4002-9221-713620028d3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:35.024155 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1114/2225] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:15:34 2022] DELETE /identity/v3/projects/dcb09d33a2664733b014a9c85deedc85 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:35.028159 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaa78c1-2703-43e9-ad16-3a60fd360d5c None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:35.039419 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaa78c1-2703-43e9-ad16-3a60fd360d5c None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:35.094595 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81cb124-2d15-43a5-a357-6f552fea766d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:35.095243 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81cb124-2d15-43a5-a357-6f552fea766d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:35.095531 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81cb124-2d15-43a5-a357-6f552fea766d admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:35.095808 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81cb124-2d15-43a5-a357-6f552fea766d admin admin] PATH_INFO: `/v3/users/cac4441034ae4f0c82ba8fbd23b3043c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:35.096936 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81cb124-2d15-43a5-a357-6f552fea766d admin admin] RBAC: Authorizing `identity:delete_user(user_id=cac4441034ae4f0c82ba8fbd23b3043c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:35.098899 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81cb124-2d15-43a5-a357-6f552fea766d admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:35.146632 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-d81cb124-2d15-43a5-a357-6f552fea766d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cac4441034ae4f0c82ba8fbd23b3043c'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:35.155823 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-d81cb124-2d15-43a5-a357-6f552fea766d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cac4441034ae4f0c82ba8fbd23b3043c'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:35.156421 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1b805d-3fe4-4002-9221-713620028d3d admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:35.157932 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-0b1b805d-3fe4-4002-9221-713620028d3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:35.163656 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-d81cb124-2d15-43a5-a357-6f552fea766d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cac4441034ae4f0c82ba8fbd23b3043c'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:35.164468 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-d81cb124-2d15-43a5-a357-6f552fea766d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cac4441034ae4f0c82ba8fbd23b3043c'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:35.176650 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-d81cb124-2d15-43a5-a357-6f552fea766d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cac4441034ae4f0c82ba8fbd23b3043c'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:35.183397 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1112/2226] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:15:35 2022] DELETE /identity/v3/users/cac4441034ae4f0c82ba8fbd23b3043c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:35.220417 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1b805d-3fe4-4002-9221-713620028d3d admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:35.278267 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aad20b-9413-482e-b5a0-41eadaa3afdc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152fd7c708114f249b9c960165f0fdfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:35.278875 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aad20b-9413-482e-b5a0-41eadaa3afdc service glance] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:35.279150 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aad20b-9413-482e-b5a0-41eadaa3afdc service glance] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:35.279457 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aad20b-9413-482e-b5a0-41eadaa3afdc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:35.281189 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-d7aad20b-9413-482e-b5a0-41eadaa3afdc service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:35.360726 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aad20b-9413-482e-b5a0-41eadaa3afdc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:35.366496 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aad20b-9413-482e-b5a0-41eadaa3afdc service glance] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:35.385982 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1115/2227] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:15:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 12:15:35.493015 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81cb124-2d15-43a5-a357-6f552fea766d admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:35.494655 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-d81cb124-2d15-43a5-a357-6f552fea766d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:35.575411 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81cb124-2d15-43a5-a357-6f552fea766d admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:35.581717 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8a9557-ad6e-495e-a638-4e986d4b5bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:35.582653 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a9557-ad6e-495e-a638-4e986d4b5bf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:35.583110 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a9557-ad6e-495e-a638-4e986d4b5bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:35.583187 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a9557-ad6e-495e-a638-4e986d4b5bf7 admin admin] PATH_INFO: `/v3/projects/5f32683bda15462dad208e6ac5ce123b` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:35.585026 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8a9557-ad6e-495e-a638-4e986d4b5bf7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f32683bda15462dad208e6ac5ce123b)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:35.587842 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8a9557-ad6e-495e-a638-4e986d4b5bf7 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:35.596064 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:35.644673 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-4f8a9557-ad6e-495e-a638-4e986d4b5bf7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f32683bda15462dad208e6ac5ce123b', 'request_id': 'req-4f8a9557-ad6e-495e-a638-4e986d4b5bf7'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:35.651769 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-4f8a9557-ad6e-495e-a638-4e986d4b5bf7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f32683bda15462dad208e6ac5ce123b', 'request_id': 'req-4f8a9557-ad6e-495e-a638-4e986d4b5bf7'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:35.659684 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-4f8a9557-ad6e-495e-a638-4e986d4b5bf7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f32683bda15462dad208e6ac5ce123b', 'request_id': 'req-4f8a9557-ad6e-495e-a638-4e986d4b5bf7'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:35.670257 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-4f8a9557-ad6e-495e-a638-4e986d4b5bf7 admin admin] The token cache is being invalidate because project 5f32683bda15462dad208e6ac5ce123b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:15:35.670549 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-4f8a9557-ad6e-495e-a638-4e986d4b5bf7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:35.671179 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-4f8a9557-ad6e-495e-a638-4e986d4b5bf7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:35.672922 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1113/2228] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:15:35 2022] DELETE /identity/v3/projects/5f32683bda15462dad208e6ac5ce123b => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:35.678013 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd3e5d-865f-48db-acef-f509f8f5b31d None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:35.678752 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd3e5d-865f-48db-acef-f509f8f5b31d None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:35.678752 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd3e5d-865f-48db-acef-f509f8f5b31d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:35.699730 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-65dd3e5d-865f-48db-acef-f509f8f5b31d None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:35.701117 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-65dd3e5d-865f-48db-acef-f509f8f5b31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:35.738762 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:35.787353 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1116/2229] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:15:35.796346 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbcd80d-4c5b-46b9-a579-85b4ac7e71bc None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:35.796634 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbcd80d-4c5b-46b9-a579-85b4ac7e71bc None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:35.797044 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbcd80d-4c5b-46b9-a579-85b4ac7e71bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:35.833421 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-cdbcd80d-4c5b-46b9-a579-85b4ac7e71bc None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:35.835519 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-cdbcd80d-4c5b-46b9-a579-85b4ac7e71bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:35.847243 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1114/2230] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:15:35.850346 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dd3e5d-865f-48db-acef-f509f8f5b31d None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:35.861739 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dd3e5d-865f-48db-acef-f509f8f5b31d None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:35.870376 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8703331f-3816-4441-8dc3-228bc7e311a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:35.871000 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8703331f-3816-4441-8dc3-228bc7e311a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:35.871312 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8703331f-3816-4441-8dc3-228bc7e311a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:35.871593 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8703331f-3816-4441-8dc3-228bc7e311a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:35.872264 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8703331f-3816-4441-8dc3-228bc7e311a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:15:35.872584 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8703331f-3816-4441-8dc3-228bc7e311a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:35.874177 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8703331f-3816-4441-8dc3-228bc7e311a1 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:35.890898 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1117/2231] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:15:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:15:35.892859 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbcd80d-4c5b-46b9-a579-85b4ac7e71bc None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:35.907593 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbcd80d-4c5b-46b9-a579-85b4ac7e71bc None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:35.913373 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb5aad6-63ff-4b75-abbe-1af2a5e4dde5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:35.914060 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5aad6-63ff-4b75-abbe-1af2a5e4dde5 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:35.914428 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5aad6-63ff-4b75-abbe-1af2a5e4dde5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:35.914752 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5aad6-63ff-4b75-abbe-1af2a5e4dde5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:35.915504 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb5aad6-63ff-4b75-abbe-1af2a5e4dde5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:35.917605 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb5aad6-63ff-4b75-abbe-1af2a5e4dde5 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:35.965589 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1115/2232] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:15:35 2022] POST /identity/v3/projects => generated 375 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:15:35.970034 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8703331f-3816-4441-8dc3-228bc7e311a1 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:35.983114 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8703331f-3816-4441-8dc3-228bc7e311a1 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:36.051338 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eda59c-4066-4b84-951f-f71c82484134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:36.051870 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eda59c-4066-4b84-951f-f71c82484134 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:36.052149 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eda59c-4066-4b84-951f-f71c82484134 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:36.052417 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eda59c-4066-4b84-951f-f71c82484134 admin admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:36.053738 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eda59c-4066-4b84-951f-f71c82484134 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:36.055844 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eda59c-4066-4b84-951f-f71c82484134 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:36.076517 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1118/2233] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:15:35 2022] POST /identity/v3/users => generated 288 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:15:36.080788 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb5aad6-63ff-4b75-abbe-1af2a5e4dde5 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:36.091415 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb5aad6-63ff-4b75-abbe-1af2a5e4dde5 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:36.096157 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5b3aef-f970-43bd-96ac-91cc4de76f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:36.096752 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b3aef-f970-43bd-96ac-91cc4de76f4c admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:36.097026 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b3aef-f970-43bd-96ac-91cc4de76f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:36.097292 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b3aef-f970-43bd-96ac-91cc4de76f4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:36.097923 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5b3aef-f970-43bd-96ac-91cc4de76f4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:36.099539 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5b3aef-f970-43bd-96ac-91cc4de76f4c admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:36.109656 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1116/2234] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:15:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:15:36.113566 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eda59c-4066-4b84-951f-f71c82484134 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:36.125043 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eda59c-4066-4b84-951f-f71c82484134 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:36.133453 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6848f9d-5cf4-44ab-b68d-f00a0dcb9a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:36.134064 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6848f9d-5cf4-44ab-b68d-f00a0dcb9a3d admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:36.134332 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6848f9d-5cf4-44ab-b68d-f00a0dcb9a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:36.134604 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6848f9d-5cf4-44ab-b68d-f00a0dcb9a3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:36.135273 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6848f9d-5cf4-44ab-b68d-f00a0dcb9a3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:36.136859 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6848f9d-5cf4-44ab-b68d-f00a0dcb9a3d admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:36.148113 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1119/2235] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:15:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:15:36.152365 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5b3aef-f970-43bd-96ac-91cc4de76f4c admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:36.162734 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5b3aef-f970-43bd-96ac-91cc4de76f4c admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:36.167716 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a2c9fc-126e-4a48-af11-d7ec434271dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:36.168383 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a2c9fc-126e-4a48-af11-d7ec434271dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:36.168662 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a2c9fc-126e-4a48-af11-d7ec434271dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:36.168940 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a2c9fc-126e-4a48-af11-d7ec434271dc admin admin] PATH_INFO: `/v3/projects/af0b038d876c43269fb5e518779c2d9c/users/8fd37f3791c64e8c8eb07237adedd7f5/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:36.184300 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a2c9fc-126e-4a48-af11-d7ec434271dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=af0b038d876c43269fb5e518779c2d9c, user_id=8fd37f3791c64e8c8eb07237adedd7f5, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:36.187039 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a2c9fc-126e-4a48-af11-d7ec434271dc admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:36.201041 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1117/2236] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:15:36 2022] PUT /identity/v3/projects/af0b038d876c43269fb5e518779c2d9c/users/8fd37f3791c64e8c8eb07237adedd7f5/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:36.211974 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09dd54-87fc-4814-a929-a1c0ff26540d None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:36.212338 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09dd54-87fc-4814-a929-a1c0ff26540d None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:36.212720 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09dd54-87fc-4814-a929-a1c0ff26540d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:36.244044 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-ad09dd54-87fc-4814-a929-a1c0ff26540d None None] MFA Rules not processed for user `8fd37f3791c64e8c8eb07237adedd7f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:36.245288 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-ad09dd54-87fc-4814-a929-a1c0ff26540d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:36.311966 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1120/2237] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:15:36.316664 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a2c9fc-126e-4a48-af11-d7ec434271dc admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:36.384640 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a2c9fc-126e-4a48-af11-d7ec434271dc admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:36.390219 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1628690-43f5-42eb-a500-128e87a8089a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:36.390825 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1628690-43f5-42eb-a500-128e87a8089a admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:36.391125 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1628690-43f5-42eb-a500-128e87a8089a admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:36.391577 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1628690-43f5-42eb-a500-128e87a8089a admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:36.392199 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1628690-43f5-42eb-a500-128e87a8089a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:36.394084 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1628690-43f5-42eb-a500-128e87a8089a admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:36.423430 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1118/2238] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:15:36 2022] POST /identity/v3/projects => generated 375 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:15:36.424642 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad09dd54-87fc-4814-a929-a1c0ff26540d None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:36.436174 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad09dd54-87fc-4814-a929-a1c0ff26540d None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:36.463386 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:36.498775 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70ca54a-c728-401e-9477-d8dd2f0d7f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:36.499418 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ca54a-c728-401e-9477-d8dd2f0d7f10 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:36.499697 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ca54a-c728-401e-9477-d8dd2f0d7f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:36.499964 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ca54a-c728-401e-9477-d8dd2f0d7f10 admin admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:36.500612 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70ca54a-c728-401e-9477-d8dd2f0d7f10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:36.502679 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70ca54a-c728-401e-9477-d8dd2f0d7f10 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:36.523782 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1121/2239] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:15:36 2022] POST /identity/v3/users => generated 294 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 12:15:36.527985 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1628690-43f5-42eb-a500-128e87a8089a admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:36.538946 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1628690-43f5-42eb-a500-128e87a8089a admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:36.543722 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce633c92-f3ab-4961-b048-93b06d08ec53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:36.544342 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce633c92-f3ab-4961-b048-93b06d08ec53 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:36.544612 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce633c92-f3ab-4961-b048-93b06d08ec53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:36.544887 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce633c92-f3ab-4961-b048-93b06d08ec53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:36.545561 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce633c92-f3ab-4961-b048-93b06d08ec53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:36.547237 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce633c92-f3ab-4961-b048-93b06d08ec53 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:36.559432 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1119/2240] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:15:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:15:36.564325 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70ca54a-c728-401e-9477-d8dd2f0d7f10 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:36.574407 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70ca54a-c728-401e-9477-d8dd2f0d7f10 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:36.578939 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dc22d2-659f-4bc5-bd5a-fc29c40f73cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:36.579609 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc22d2-659f-4bc5-bd5a-fc29c40f73cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:36.579884 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc22d2-659f-4bc5-bd5a-fc29c40f73cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:36.580151 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc22d2-659f-4bc5-bd5a-fc29c40f73cf admin admin] PATH_INFO: `/v3/projects/6c748e6f935247ac99617e6b1845277b/users/356867146ad4406caa2e88d58bee8a28/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:36.594963 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dc22d2-659f-4bc5-bd5a-fc29c40f73cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c748e6f935247ac99617e6b1845277b, user_id=356867146ad4406caa2e88d58bee8a28, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:36.598129 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dc22d2-659f-4bc5-bd5a-fc29c40f73cf admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:36.614009 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1122/2241] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:15:36 2022] PUT /identity/v3/projects/6c748e6f935247ac99617e6b1845277b/users/356867146ad4406caa2e88d58bee8a28/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:36.619038 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce633c92-f3ab-4961-b048-93b06d08ec53 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:36.625575 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:36.629745 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce633c92-f3ab-4961-b048-93b06d08ec53 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:36.689440 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce16888-7471-41b1-b239-de32c58eee9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:36.690045 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce16888-7471-41b1-b239-de32c58eee9b admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:36.690328 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce16888-7471-41b1-b239-de32c58eee9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:36.690604 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce16888-7471-41b1-b239-de32c58eee9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:36.691261 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce16888-7471-41b1-b239-de32c58eee9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:36.692850 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce16888-7471-41b1-b239-de32c58eee9b admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:36.703415 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1120/2242] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:15:36 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:15:36.707717 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dc22d2-659f-4bc5-bd5a-fc29c40f73cf admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:36.718669 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dc22d2-659f-4bc5-bd5a-fc29c40f73cf admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:36.723633 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b298553-75a0-47b8-ab2f-1205db6f61bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:36.724315 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b298553-75a0-47b8-ab2f-1205db6f61bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:36.724614 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b298553-75a0-47b8-ab2f-1205db6f61bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:36.725458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b298553-75a0-47b8-ab2f-1205db6f61bf admin admin] PATH_INFO: `/v3/projects/6c748e6f935247ac99617e6b1845277b/users/356867146ad4406caa2e88d58bee8a28/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:36.727549 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b298553-75a0-47b8-ab2f-1205db6f61bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c748e6f935247ac99617e6b1845277b, user_id=356867146ad4406caa2e88d58bee8a28, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:36.730387 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b298553-75a0-47b8-ab2f-1205db6f61bf admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:36.743492 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1123/2243] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:15:36 2022] PUT /identity/v3/projects/6c748e6f935247ac99617e6b1845277b/users/356867146ad4406caa2e88d58bee8a28/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:36.755134 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e85cc8-f03f-4bd1-87de-90a647245f1f None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:36.755453 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e85cc8-f03f-4bd1-87de-90a647245f1f None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:36.755757 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e85cc8-f03f-4bd1-87de-90a647245f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:36.785352 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-35e85cc8-f03f-4bd1-87de-90a647245f1f None None] MFA Rules not processed for user `356867146ad4406caa2e88d58bee8a28`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:36.786741 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-35e85cc8-f03f-4bd1-87de-90a647245f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:36.857160 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1121/2244] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:36 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:15:36.868903 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b298553-75a0-47b8-ab2f-1205db6f61bf admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:36.870617 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-3b298553-75a0-47b8-ab2f-1205db6f61bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:36.931693 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b298553-75a0-47b8-ab2f-1205db6f61bf admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:36.932725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e85cc8-f03f-4bd1-87de-90a647245f1f None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:36.933903 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d3684f-9d04-4231-a419-3703d91b06bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:36.934135 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-35e85cc8-f03f-4bd1-87de-90a647245f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:36.934531 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d3684f-9d04-4231-a419-3703d91b06bb service cinder] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:36.934807 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d3684f-9d04-4231-a419-3703d91b06bb service cinder] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:36.935216 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d3684f-9d04-4231-a419-3703d91b06bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:37.011494 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e85cc8-f03f-4bd1-87de-90a647245f1f None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:37.014493 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d3684f-9d04-4231-a419-3703d91b06bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:37.016011 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d3684f-9d04-4231-a419-3703d91b06bb service cinder] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:37.018659 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a60b4eb-812a-40f1-8259-fd63c819d23b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:37.019529 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a60b4eb-812a-40f1-8259-fd63c819d23b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:37.019808 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a60b4eb-812a-40f1-8259-fd63c819d23b admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:37.020083 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a60b4eb-812a-40f1-8259-fd63c819d23b admin admin] PATH_INFO: `/v3/users/db1e513c022c4b369ae9ce73a2f5f52e` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:37.021205 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a60b4eb-812a-40f1-8259-fd63c819d23b admin admin] RBAC: Authorizing `identity:delete_user(user_id=db1e513c022c4b369ae9ce73a2f5f52e)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:37.023360 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a60b4eb-812a-40f1-8259-fd63c819d23b admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:37.025292 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1124/2245] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:15:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:15:37.058268 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:37.089907 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-5a60b4eb-812a-40f1-8259-fd63c819d23b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db1e513c022c4b369ae9ce73a2f5f52e'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:37.102512 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-5a60b4eb-812a-40f1-8259-fd63c819d23b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db1e513c022c4b369ae9ce73a2f5f52e'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:37.111976 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-5a60b4eb-812a-40f1-8259-fd63c819d23b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db1e513c022c4b369ae9ce73a2f5f52e'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:37.112929 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-5a60b4eb-812a-40f1-8259-fd63c819d23b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db1e513c022c4b369ae9ce73a2f5f52e'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:37.132463 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-5a60b4eb-812a-40f1-8259-fd63c819d23b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db1e513c022c4b369ae9ce73a2f5f52e'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:37.139841 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1122/2246] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:15:36 2022] DELETE /identity/v3/users/db1e513c022c4b369ae9ce73a2f5f52e => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 12:15:37.144261 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa739f8-f0cc-487c-a8a7-fa9394e5957a None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:37.144551 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa739f8-f0cc-487c-a8a7-fa9394e5957a None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:37.144824 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa739f8-f0cc-487c-a8a7-fa9394e5957a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:37.180658 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-aaa739f8-f0cc-487c-a8a7-fa9394e5957a None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:37.182023 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-aaa739f8-f0cc-487c-a8a7-fa9394e5957a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:37.223371 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:37.264545 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1125/2247] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:15:37.275467 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a60b4eb-812a-40f1-8259-fd63c819d23b admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:37.277691 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-5a60b4eb-812a-40f1-8259-fd63c819d23b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:37.351492 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a60b4eb-812a-40f1-8259-fd63c819d23b admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:37.356135 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b7afe6-085a-4345-b7fb-a7ae1c7436b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:37.357128 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b7afe6-085a-4345-b7fb-a7ae1c7436b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:37.357416 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b7afe6-085a-4345-b7fb-a7ae1c7436b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:37.357697 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b7afe6-085a-4345-b7fb-a7ae1c7436b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:37.368443 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b7afe6-085a-4345-b7fb-a7ae1c7436b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:37.369891 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b7afe6-085a-4345-b7fb-a7ae1c7436b2 service neutron] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:37.382049 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1123/2248] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:15:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:15:37.672123 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa739f8-f0cc-487c-a8a7-fa9394e5957a None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:37.675402 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-aaa739f8-f0cc-487c-a8a7-fa9394e5957a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:37.689364 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa739f8-f0cc-487c-a8a7-fa9394e5957a None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:37.695239 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1385755-0f7e-4e95-be66-74b1c70a9112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:37.696038 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1385755-0f7e-4e95-be66-74b1c70a9112 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:37.696318 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1385755-0f7e-4e95-be66-74b1c70a9112 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:37.696591 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1385755-0f7e-4e95-be66-74b1c70a9112 admin admin] PATH_INFO: `/v3/projects/6315476c549a4763993b43485ac34917` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:37.697790 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1385755-0f7e-4e95-be66-74b1c70a9112 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6315476c549a4763993b43485ac34917)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:37.699729 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1385755-0f7e-4e95-be66-74b1c70a9112 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:37.751035 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-c1385755-0f7e-4e95-be66-74b1c70a9112 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6315476c549a4763993b43485ac34917', 'request_id': 'req-c1385755-0f7e-4e95-be66-74b1c70a9112'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:37.758577 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-c1385755-0f7e-4e95-be66-74b1c70a9112 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6315476c549a4763993b43485ac34917', 'request_id': 'req-c1385755-0f7e-4e95-be66-74b1c70a9112'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:37.767491 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-c1385755-0f7e-4e95-be66-74b1c70a9112 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6315476c549a4763993b43485ac34917', 'request_id': 'req-c1385755-0f7e-4e95-be66-74b1c70a9112'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:37.780112 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-c1385755-0f7e-4e95-be66-74b1c70a9112 admin admin] The token cache is being invalidate because project 6315476c549a4763993b43485ac34917 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81712) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:15:37.780401 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-c1385755-0f7e-4e95-be66-74b1c70a9112 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:37.781048 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-c1385755-0f7e-4e95-be66-74b1c70a9112 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:15:37.782520 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1126/2249] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:15:37 2022] DELETE /identity/v3/projects/6315476c549a4763993b43485ac34917 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:37.787789 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d8a71b-88a2-4995-b86e-25a6bceafe50 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:37.788083 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d8a71b-88a2-4995-b86e-25a6bceafe50 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:37.788345 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d8a71b-88a2-4995-b86e-25a6bceafe50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:37.807811 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-f2d8a71b-88a2-4995-b86e-25a6bceafe50 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:37.809001 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-f2d8a71b-88a2-4995-b86e-25a6bceafe50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:37.867552 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1124/2250] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:15:37.876471 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6751ee0c-ebb7-41c0-820b-ac73316fc0a4 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:37.876687 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6751ee0c-ebb7-41c0-820b-ac73316fc0a4 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:37.876964 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6751ee0c-ebb7-41c0-820b-ac73316fc0a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:37.896147 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-6751ee0c-ebb7-41c0-820b-ac73316fc0a4 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:37.897591 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-6751ee0c-ebb7-41c0-820b-ac73316fc0a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:37.905290 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1127/2251] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:15:37.909487 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d8a71b-88a2-4995-b86e-25a6bceafe50 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:37.919560 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d8a71b-88a2-4995-b86e-25a6bceafe50 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:37.924036 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e8ea3b-1471-487b-8d09-94628076aaea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:37.924627 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e8ea3b-1471-487b-8d09-94628076aaea admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:37.924906 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e8ea3b-1471-487b-8d09-94628076aaea admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:37.925174 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e8ea3b-1471-487b-8d09-94628076aaea admin admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:37.925806 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e8ea3b-1471-487b-8d09-94628076aaea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:15:37.926103 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e8ea3b-1471-487b-8d09-94628076aaea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:37.927722 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e8ea3b-1471-487b-8d09-94628076aaea admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:37.940613 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1125/2252] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:15:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:15:37.945560 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6751ee0c-ebb7-41c0-820b-ac73316fc0a4 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:37.956402 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6751ee0c-ebb7-41c0-820b-ac73316fc0a4 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:37.961680 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2b16f1-8323-465f-98a9-66b2749c9386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:37.962331 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b16f1-8323-465f-98a9-66b2749c9386 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:37.962609 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b16f1-8323-465f-98a9-66b2749c9386 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:37.963090 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b16f1-8323-465f-98a9-66b2749c9386 admin admin] PATH_INFO: `/v3/projects` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:37.963911 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2b16f1-8323-465f-98a9-66b2749c9386 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:37.966084 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2b16f1-8323-465f-98a9-66b2749c9386 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:37.996190 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1128/2253] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:15:37 2022] POST /identity/v3/projects => generated 379 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:15:38.000277 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e8ea3b-1471-487b-8d09-94628076aaea admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:38.010347 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e8ea3b-1471-487b-8d09-94628076aaea admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:38.048455 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:38.079812 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87144cb9-b48f-4dd7-be3d-ff608ab2b2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:38.080432 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87144cb9-b48f-4dd7-be3d-ff608ab2b2ca admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:38.080713 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87144cb9-b48f-4dd7-be3d-ff608ab2b2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:38.080989 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87144cb9-b48f-4dd7-be3d-ff608ab2b2ca admin admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:38.081627 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87144cb9-b48f-4dd7-be3d-ff608ab2b2ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:38.083590 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87144cb9-b48f-4dd7-be3d-ff608ab2b2ca admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:38.105220 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1126/2254] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:15:37 2022] POST /identity/v3/users => generated 290 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:15:38.109361 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2b16f1-8323-465f-98a9-66b2749c9386 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:38.120715 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2b16f1-8323-465f-98a9-66b2749c9386 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:38.125822 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0dcf3d-7b10-432f-abdb-afcc3c346d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:38.126435 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0dcf3d-7b10-432f-abdb-afcc3c346d3f admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:38.126761 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0dcf3d-7b10-432f-abdb-afcc3c346d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:38.126978 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0dcf3d-7b10-432f-abdb-afcc3c346d3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:38.127663 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0dcf3d-7b10-432f-abdb-afcc3c346d3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:38.129254 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0dcf3d-7b10-432f-abdb-afcc3c346d3f admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:38.139759 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1129/2255] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:15:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:15:38.143520 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87144cb9-b48f-4dd7-be3d-ff608ab2b2ca admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:38.153459 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87144cb9-b48f-4dd7-be3d-ff608ab2b2ca admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:38.158158 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269894c6-afd1-42c3-9ebc-57e7d7e7ae85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:38.158749 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269894c6-afd1-42c3-9ebc-57e7d7e7ae85 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:38.159031 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269894c6-afd1-42c3-9ebc-57e7d7e7ae85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:38.159343 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269894c6-afd1-42c3-9ebc-57e7d7e7ae85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:38.160017 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269894c6-afd1-42c3-9ebc-57e7d7e7ae85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:38.161593 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269894c6-afd1-42c3-9ebc-57e7d7e7ae85 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:38.172745 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1127/2256] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:15:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:15:38.176655 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0dcf3d-7b10-432f-abdb-afcc3c346d3f admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:38.187265 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0dcf3d-7b10-432f-abdb-afcc3c346d3f admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:38.192027 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deba3921-4bb8-418e-89f7-d6333550904d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:38.192639 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deba3921-4bb8-418e-89f7-d6333550904d admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:38.192920 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deba3921-4bb8-418e-89f7-d6333550904d admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:38.193193 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deba3921-4bb8-418e-89f7-d6333550904d admin admin] PATH_INFO: `/v3/projects/293ca4bc65af4f25afb8456f32378173/users/2b51cd55973a419cad6311609395bb89/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:38.208257 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deba3921-4bb8-418e-89f7-d6333550904d admin admin] RBAC: Authorizing `identity:create_grant(project_id=293ca4bc65af4f25afb8456f32378173, user_id=2b51cd55973a419cad6311609395bb89, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:38.211676 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deba3921-4bb8-418e-89f7-d6333550904d admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:38.224578 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1130/2257] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:15:38 2022] PUT /identity/v3/projects/293ca4bc65af4f25afb8456f32378173/users/2b51cd55973a419cad6311609395bb89/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:38.230075 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89515c5e-7514-4b88-8c3a-061ddad9c73d None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:38.230384 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89515c5e-7514-4b88-8c3a-061ddad9c73d None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:38.230685 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89515c5e-7514-4b88-8c3a-061ddad9c73d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:38.249136 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-89515c5e-7514-4b88-8c3a-061ddad9c73d None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:38.250330 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-89515c5e-7514-4b88-8c3a-061ddad9c73d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:38.319006 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1128/2258] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:15:38.328565 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deba3921-4bb8-418e-89f7-d6333550904d admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:38.330308 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-deba3921-4bb8-418e-89f7-d6333550904d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:38.407501 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deba3921-4bb8-418e-89f7-d6333550904d admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:38.411437 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b701e67-7761-4cb9-87a5-62587b35a6cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:38.412059 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b701e67-7761-4cb9-87a5-62587b35a6cd service neutron] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:38.413017 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b701e67-7761-4cb9-87a5-62587b35a6cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:38.417800 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b701e67-7761-4cb9-87a5-62587b35a6cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:38.433725 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b701e67-7761-4cb9-87a5-62587b35a6cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:38.435631 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b701e67-7761-4cb9-87a5-62587b35a6cd service neutron] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:38.442461 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:38.456961 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1131/2259] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:15:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:15:42.757756 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518d7873-09f8-4b71-9a0b-3c3a78449f9d None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:42.758226 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518d7873-09f8-4b71-9a0b-3c3a78449f9d None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:42.758303 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518d7873-09f8-4b71-9a0b-3c3a78449f9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:42.808258 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-518d7873-09f8-4b71-9a0b-3c3a78449f9d None None] MFA Rules not processed for user `2b51cd55973a419cad6311609395bb89`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:42.809466 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-518d7873-09f8-4b71-9a0b-3c3a78449f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:42.812460 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b701e67-7761-4cb9-87a5-62587b35a6cd service neutron] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:42.813936 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-1b701e67-7761-4cb9-87a5-62587b35a6cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:42.882089 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b701e67-7761-4cb9-87a5-62587b35a6cd service neutron] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:42.883304 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1129/2260] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:15:42.885727 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9150c218-4fa1-4f57-8a30-d86a9d8bc839 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:42.886402 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150c218-4fa1-4f57-8a30-d86a9d8bc839 service cinder] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:42.886691 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150c218-4fa1-4f57-8a30-d86a9d8bc839 service cinder] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:42.887015 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150c218-4fa1-4f57-8a30-d86a9d8bc839 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:42.887627 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518d7873-09f8-4b71-9a0b-3c3a78449f9d None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:42.889162 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-9150c218-4fa1-4f57-8a30-d86a9d8bc839 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:42.899187 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518d7873-09f8-4b71-9a0b-3c3a78449f9d None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:42.905740 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bc2032-253d-43c9-87f4-73566f5bd96a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:42.906352 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bc2032-253d-43c9-87f4-73566f5bd96a admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:42.906630 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bc2032-253d-43c9-87f4-73566f5bd96a admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:42.906904 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bc2032-253d-43c9-87f4-73566f5bd96a admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:42.907630 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bc2032-253d-43c9-87f4-73566f5bd96a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:42.914258 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bc2032-253d-43c9-87f4-73566f5bd96a admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:42.948930 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1130/2261] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:15:42 2022] POST /identity/v3/projects => generated 377 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:15:42.952864 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bc2032-253d-43c9-87f4-73566f5bd96a admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:42.968097 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9150c218-4fa1-4f57-8a30-d86a9d8bc839 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:42.969557 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9150c218-4fa1-4f57-8a30-d86a9d8bc839 service cinder] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:42.983257 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1132/2262] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:15:42 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:15:42.983450 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:43.075273 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bc2032-253d-43c9-87f4-73566f5bd96a admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:43.093640 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7ad8e0-9f4d-4e4a-8e48-94b190e95f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:43.094399 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ad8e0-9f4d-4e4a-8e48-94b190e95f23 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:43.094845 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ad8e0-9f4d-4e4a-8e48-94b190e95f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:43.095343 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ad8e0-9f4d-4e4a-8e48-94b190e95f23 admin admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:43.096044 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7ad8e0-9f4d-4e4a-8e48-94b190e95f23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:43.098219 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7ad8e0-9f4d-4e4a-8e48-94b190e95f23 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:43.135679 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1131/2263] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:15:42 2022] POST /identity/v3/users => generated 295 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 12:15:43.136996 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9150c218-4fa1-4f57-8a30-d86a9d8bc839 service cinder] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:43.149565 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9150c218-4fa1-4f57-8a30-d86a9d8bc839 service cinder] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:43.154608 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8063dcd-5923-4582-83cf-2a6794f43fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:43.155462 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8063dcd-5923-4582-83cf-2a6794f43fb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:43.155896 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8063dcd-5923-4582-83cf-2a6794f43fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:43.156325 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8063dcd-5923-4582-83cf-2a6794f43fb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:43.157377 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8063dcd-5923-4582-83cf-2a6794f43fb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:43.159160 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8063dcd-5923-4582-83cf-2a6794f43fb4 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:43.173041 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1133/2264] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:15:43.177112 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7ad8e0-9f4d-4e4a-8e48-94b190e95f23 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:43.188329 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7ad8e0-9f4d-4e4a-8e48-94b190e95f23 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:43.195467 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f581c431-3a4c-479c-a48f-0cd7b5d6f2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:43.196157 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f581c431-3a4c-479c-a48f-0cd7b5d6f2de admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:43.196523 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f581c431-3a4c-479c-a48f-0cd7b5d6f2de admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:43.196709 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f581c431-3a4c-479c-a48f-0cd7b5d6f2de admin admin] PATH_INFO: `/v3/projects/bf6dc52acadd44a493ed8e386e803a99/users/194922a3d1974964997c4905dcf85359/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:43.214123 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f581c431-3a4c-479c-a48f-0cd7b5d6f2de admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf6dc52acadd44a493ed8e386e803a99, user_id=194922a3d1974964997c4905dcf85359, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:43.216977 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f581c431-3a4c-479c-a48f-0cd7b5d6f2de admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:43.235745 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1132/2265] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:15:43 2022] PUT /identity/v3/projects/bf6dc52acadd44a493ed8e386e803a99/users/194922a3d1974964997c4905dcf85359/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 12:15:43.239708 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8063dcd-5923-4582-83cf-2a6794f43fb4 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:43.251010 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8063dcd-5923-4582-83cf-2a6794f43fb4 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:43.338805 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11a75e9-fedc-4a07-8d3a-912a0b99a268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:43.339597 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11a75e9-fedc-4a07-8d3a-912a0b99a268 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:43.339976 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11a75e9-fedc-4a07-8d3a-912a0b99a268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:43.340432 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11a75e9-fedc-4a07-8d3a-912a0b99a268 admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:43.341098 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11a75e9-fedc-4a07-8d3a-912a0b99a268 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:43.343297 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11a75e9-fedc-4a07-8d3a-912a0b99a268 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:43.362592 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1134/2266] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:15:43.363884 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f581c431-3a4c-479c-a48f-0cd7b5d6f2de admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:43.379281 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f581c431-3a4c-479c-a48f-0cd7b5d6f2de admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:43.395563 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f52710b-62af-4b44-afdb-0297cb7880d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:43.396301 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52710b-62af-4b44-afdb-0297cb7880d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:43.397135 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52710b-62af-4b44-afdb-0297cb7880d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:43.397468 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52710b-62af-4b44-afdb-0297cb7880d2 admin admin] PATH_INFO: `/v3/projects/bf6dc52acadd44a493ed8e386e803a99/users/194922a3d1974964997c4905dcf85359/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:43.413261 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f52710b-62af-4b44-afdb-0297cb7880d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf6dc52acadd44a493ed8e386e803a99, user_id=194922a3d1974964997c4905dcf85359, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:43.413261 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f52710b-62af-4b44-afdb-0297cb7880d2 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:43.454416 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1133/2267] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:15:43 2022] PUT /identity/v3/projects/bf6dc52acadd44a493ed8e386e803a99/users/194922a3d1974964997c4905dcf85359/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:15:48.419137 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7053e0-29d1-4bf5-b8aa-b65477214425 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:48.420031 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7053e0-29d1-4bf5-b8aa-b65477214425 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:48.420498 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7053e0-29d1-4bf5-b8aa-b65477214425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:48.480475 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-0c7053e0-29d1-4bf5-b8aa-b65477214425 None None] MFA Rules not processed for user `194922a3d1974964997c4905dcf85359`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:15:48.481835 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-0c7053e0-29d1-4bf5-b8aa-b65477214425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:48.521351 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:48.587406 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1135/2268] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:15:48 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:15:48.601384 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f52710b-62af-4b44-afdb-0297cb7880d2 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:15:48.605247 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-4f52710b-62af-4b44-afdb-0297cb7880d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:15:48.705447 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f52710b-62af-4b44-afdb-0297cb7880d2 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:15:48.712176 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b66a66-5a28-4a09-bb86-76c279d43be8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aacd7d6d17e4f10ba5c9107a12beafa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:15:48.712791 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b66a66-5a28-4a09-bb86-76c279d43be8 service nova] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:15:48.713067 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b66a66-5a28-4a09-bb86-76c279d43be8 service nova] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:15:48.713471 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b66a66-5a28-4a09-bb86-76c279d43be8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:15:48.795116 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b66a66-5a28-4a09-bb86-76c279d43be8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:15:48.796816 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b66a66-5a28-4a09-bb86-76c279d43be8 service nova] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:15:48.807907 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1134/2269] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:15:48 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:15:58.580012 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=19 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:15:58.803078 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=90,keystone:INSERT=5 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:16:17.516836 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7053e0-29d1-4bf5-b8aa-b65477214425 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:17.518340 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-0c7053e0-29d1-4bf5-b8aa-b65477214425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:17.589705 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7053e0-29d1-4bf5-b8aa-b65477214425 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:17.595211 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9be35a-cf67-4f0a-9967-5ed08a4f85ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:17.595931 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9be35a-cf67-4f0a-9967-5ed08a4f85ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:17.596212 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9be35a-cf67-4f0a-9967-5ed08a4f85ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:17.596480 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9be35a-cf67-4f0a-9967-5ed08a4f85ba admin admin] PATH_INFO: `/v3/users/53c54f06cbb54dd48e3d99fae89fc107` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:17.597665 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9be35a-cf67-4f0a-9967-5ed08a4f85ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=53c54f06cbb54dd48e3d99fae89fc107)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:17.599743 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9be35a-cf67-4f0a-9967-5ed08a4f85ba admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:17.652087 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b66a66-5a28-4a09-bb86-76c279d43be8 service nova] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:17.656013 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-b2b66a66-5a28-4a09-bb86-76c279d43be8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:17.670082 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-bd9be35a-cf67-4f0a-9967-5ed08a4f85ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53c54f06cbb54dd48e3d99fae89fc107'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:17.671709 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b66a66-5a28-4a09-bb86-76c279d43be8 service nova] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:17.679598 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-bd9be35a-cf67-4f0a-9967-5ed08a4f85ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53c54f06cbb54dd48e3d99fae89fc107'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:17.680955 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-bd9be35a-cf67-4f0a-9967-5ed08a4f85ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53c54f06cbb54dd48e3d99fae89fc107'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:17.686054 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f48b73a-9114-442a-acd6-9dd768ec779a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:17.686701 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f48b73a-9114-442a-acd6-9dd768ec779a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:17.686985 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f48b73a-9114-442a-acd6-9dd768ec779a admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:17.687447 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f48b73a-9114-442a-acd6-9dd768ec779a admin admin] PATH_INFO: `/v3/users/116651577f2e49e2a4fc6c5ad339d358` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:17.689174 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f48b73a-9114-442a-acd6-9dd768ec779a admin admin] RBAC: Authorizing `identity:delete_user(user_id=116651577f2e49e2a4fc6c5ad339d358)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:17.690037 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-bd9be35a-cf67-4f0a-9967-5ed08a4f85ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53c54f06cbb54dd48e3d99fae89fc107'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:17.690668 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-bd9be35a-cf67-4f0a-9967-5ed08a4f85ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53c54f06cbb54dd48e3d99fae89fc107'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:17.693513 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f48b73a-9114-442a-acd6-9dd768ec779a admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:17.722363 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1136/2270] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:16:17 2022] DELETE /identity/v3/users/53c54f06cbb54dd48e3d99fae89fc107 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:16:17.727953 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb27fd3-5fdc-45bf-af98-f946aa5ab49a None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:17.728248 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb27fd3-5fdc-45bf-af98-f946aa5ab49a None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:17.728526 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb27fd3-5fdc-45bf-af98-f946aa5ab49a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:17.751061 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-dfb27fd3-5fdc-45bf-af98-f946aa5ab49a None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:16:17.752092 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-dfb27fd3-5fdc-45bf-af98-f946aa5ab49a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:17.768590 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-1f48b73a-9114-442a-acd6-9dd768ec779a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '116651577f2e49e2a4fc6c5ad339d358'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:17.779503 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-1f48b73a-9114-442a-acd6-9dd768ec779a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '116651577f2e49e2a4fc6c5ad339d358'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:17.786756 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-1f48b73a-9114-442a-acd6-9dd768ec779a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '116651577f2e49e2a4fc6c5ad339d358'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:17.787420 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-1f48b73a-9114-442a-acd6-9dd768ec779a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '116651577f2e49e2a4fc6c5ad339d358'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:17.800588 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-1f48b73a-9114-442a-acd6-9dd768ec779a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '116651577f2e49e2a4fc6c5ad339d358'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:17.806584 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1135/2271] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:16:17 2022] DELETE /identity/v3/users/116651577f2e49e2a4fc6c5ad339d358 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:16:17.810332 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f48b73a-9114-442a-acd6-9dd768ec779a admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:17.811669 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-1f48b73a-9114-442a-acd6-9dd768ec779a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:17.815596 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1137/2272] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:16:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:16:17.816818 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b02290-dbec-46d1-b91e-c27c9d2b13c6 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:17.817255 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b02290-dbec-46d1-b91e-c27c9d2b13c6 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:17.817529 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b02290-dbec-46d1-b91e-c27c9d2b13c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:17.837129 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-b9b02290-dbec-46d1-b91e-c27c9d2b13c6 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:16:17.838548 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-b9b02290-dbec-46d1-b91e-c27c9d2b13c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:17.854097 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:16:17.884836 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f48b73a-9114-442a-acd6-9dd768ec779a admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:17.889843 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd8dfa3-8538-4ed1-8a21-173860efeff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:17.890893 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8dfa3-8538-4ed1-8a21-173860efeff5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:17.891186 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8dfa3-8538-4ed1-8a21-173860efeff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:17.891485 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8dfa3-8538-4ed1-8a21-173860efeff5 admin admin] PATH_INFO: `/v3/users/8fd37f3791c64e8c8eb07237adedd7f5` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:17.892603 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd8dfa3-8538-4ed1-8a21-173860efeff5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fd37f3791c64e8c8eb07237adedd7f5)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:17.894825 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd8dfa3-8538-4ed1-8a21-173860efeff5 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:17.907652 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1138/2273] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:16:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:16:17.908835 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b02290-dbec-46d1-b91e-c27c9d2b13c6 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:17.910256 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-b9b02290-dbec-46d1-b91e-c27c9d2b13c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:17.943601 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-ecd8dfa3-8538-4ed1-8a21-173860efeff5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fd37f3791c64e8c8eb07237adedd7f5'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:17.952393 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-ecd8dfa3-8538-4ed1-8a21-173860efeff5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fd37f3791c64e8c8eb07237adedd7f5'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:17.962133 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-ecd8dfa3-8538-4ed1-8a21-173860efeff5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fd37f3791c64e8c8eb07237adedd7f5'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:17.963115 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-ecd8dfa3-8538-4ed1-8a21-173860efeff5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fd37f3791c64e8c8eb07237adedd7f5'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:17.972299 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:16:17.978742 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b02290-dbec-46d1-b91e-c27c9d2b13c6 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:17.978929 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-ecd8dfa3-8538-4ed1-8a21-173860efeff5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fd37f3791c64e8c8eb07237adedd7f5'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:17.982934 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a295bb10-e4fe-4a4d-995d-f09683364eca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:17.983649 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a295bb10-e4fe-4a4d-995d-f09683364eca service neutron] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:17.983923 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a295bb10-e4fe-4a4d-995d-f09683364eca service neutron] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:17.984251 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a295bb10-e4fe-4a4d-995d-f09683364eca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:17.986038 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-a295bb10-e4fe-4a4d-995d-f09683364eca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:17.990705 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1136/2274] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:16:17 2022] DELETE /identity/v3/users/8fd37f3791c64e8c8eb07237adedd7f5 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:16:17.990705 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd8dfa3-8538-4ed1-8a21-173860efeff5 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:17.992409 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-ecd8dfa3-8538-4ed1-8a21-173860efeff5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:18.005347 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a295bb10-e4fe-4a4d-995d-f09683364eca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:18.006930 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a295bb10-e4fe-4a4d-995d-f09683364eca service neutron] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:18.017676 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1139/2275] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:16:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:16:18.018891 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec95aef9-1567-41f3-a072-8b411bc3ef8b None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:18.019173 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec95aef9-1567-41f3-a072-8b411bc3ef8b None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:18.019477 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec95aef9-1567-41f3-a072-8b411bc3ef8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:18.053958 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-ec95aef9-1567-41f3-a072-8b411bc3ef8b None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:16:18.057906 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-ec95aef9-1567-41f3-a072-8b411bc3ef8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:18.061868 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd8dfa3-8538-4ed1-8a21-173860efeff5 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:18.067304 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc5af18-6e14-4beb-bc70-a1e997c71aff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:18.068105 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc5af18-6e14-4beb-bc70-a1e997c71aff service neutron] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:18.068392 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc5af18-6e14-4beb-bc70-a1e997c71aff service neutron] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:18.068717 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc5af18-6e14-4beb-bc70-a1e997c71aff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:18.071422 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-2dc5af18-6e14-4beb-bc70-a1e997c71aff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:18.143020 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1140/2276] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:16:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:16:18.158833 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec95aef9-1567-41f3-a072-8b411bc3ef8b None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:18.173228 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec95aef9-1567-41f3-a072-8b411bc3ef8b None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:18.179259 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c142a3aa-a1e7-424d-888d-200f5d076eff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:18.181425 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c142a3aa-a1e7-424d-888d-200f5d076eff service neutron] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:18.181633 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc5af18-6e14-4beb-bc70-a1e997c71aff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:18.182092 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c142a3aa-a1e7-424d-888d-200f5d076eff service neutron] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:18.182562 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c142a3aa-a1e7-424d-888d-200f5d076eff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:18.183350 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc5af18-6e14-4beb-bc70-a1e997c71aff service neutron] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:18.197808 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1137/2277] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:16:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:16:18.198061 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c142a3aa-a1e7-424d-888d-200f5d076eff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:18.200199 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c142a3aa-a1e7-424d-888d-200f5d076eff service neutron] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:18.211966 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1141/2278] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:16:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 12:16:18.817698 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc5af18-6e14-4beb-bc70-a1e997c71aff service neutron] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:18.819903 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-2dc5af18-6e14-4beb-bc70-a1e997c71aff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:18.832982 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc5af18-6e14-4beb-bc70-a1e997c71aff service neutron] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:18.838571 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4607f9-e148-43ff-8c46-ea4a23b16852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:18.839427 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4607f9-e148-43ff-8c46-ea4a23b16852 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:18.839823 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4607f9-e148-43ff-8c46-ea4a23b16852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:18.840157 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4607f9-e148-43ff-8c46-ea4a23b16852 admin admin] PATH_INFO: `/v3/projects/65d237e0efb8470caf0b20790088afda` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:18.841534 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4607f9-e148-43ff-8c46-ea4a23b16852 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65d237e0efb8470caf0b20790088afda)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:18.843666 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4607f9-e148-43ff-8c46-ea4a23b16852 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:18.929504 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-3f4607f9-e148-43ff-8c46-ea4a23b16852 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65d237e0efb8470caf0b20790088afda', 'request_id': 'req-3f4607f9-e148-43ff-8c46-ea4a23b16852'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:18.941683 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-3f4607f9-e148-43ff-8c46-ea4a23b16852 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65d237e0efb8470caf0b20790088afda', 'request_id': 'req-3f4607f9-e148-43ff-8c46-ea4a23b16852'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:18.953852 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-3f4607f9-e148-43ff-8c46-ea4a23b16852 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65d237e0efb8470caf0b20790088afda', 'request_id': 'req-3f4607f9-e148-43ff-8c46-ea4a23b16852'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:18.971127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-3f4607f9-e148-43ff-8c46-ea4a23b16852 admin admin] The token cache is being invalidate because project 65d237e0efb8470caf0b20790088afda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:16:18.971515 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-3f4607f9-e148-43ff-8c46-ea4a23b16852 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:18.972560 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-3f4607f9-e148-43ff-8c46-ea4a23b16852 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:18.974299 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1138/2279] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:16:18 2022] DELETE /identity/v3/projects/65d237e0efb8470caf0b20790088afda => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:16:18.979548 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205300cf-ae2d-4744-8642-b336e1b13831 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:18.979995 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205300cf-ae2d-4744-8642-b336e1b13831 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:18.980309 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205300cf-ae2d-4744-8642-b336e1b13831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:19.001891 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-205300cf-ae2d-4744-8642-b336e1b13831 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:16:19.003301 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-205300cf-ae2d-4744-8642-b336e1b13831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:19.060304 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4607f9-e148-43ff-8c46-ea4a23b16852 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:19.064125 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-3f4607f9-e148-43ff-8c46-ea4a23b16852 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:19.071768 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:16:19.097914 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1142/2280] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:16:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:16:19.098997 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205300cf-ae2d-4744-8642-b336e1b13831 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:19.100632 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-205300cf-ae2d-4744-8642-b336e1b13831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:19.106312 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:16:19.121387 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205300cf-ae2d-4744-8642-b336e1b13831 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:19.129148 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b34b6-9a9c-4f94-81c0-bf28ba712492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:19.130006 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b34b6-9a9c-4f94-81c0-bf28ba712492 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:19.131943 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b34b6-9a9c-4f94-81c0-bf28ba712492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:19.143413 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b34b6-9a9c-4f94-81c0-bf28ba712492 admin admin] PATH_INFO: `/v3/projects/af0b038d876c43269fb5e518779c2d9c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:19.147446 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41b34b6-9a9c-4f94-81c0-bf28ba712492 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af0b038d876c43269fb5e518779c2d9c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:19.149578 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41b34b6-9a9c-4f94-81c0-bf28ba712492 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:19.169395 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4607f9-e148-43ff-8c46-ea4a23b16852 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:19.177340 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cfdd4f-b75c-444f-b094-53d324db4ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:19.178001 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cfdd4f-b75c-444f-b094-53d324db4ae0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:19.178283 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cfdd4f-b75c-444f-b094-53d324db4ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:19.178560 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cfdd4f-b75c-444f-b094-53d324db4ae0 admin admin] PATH_INFO: `/v3/projects/85b9d4e830684dca9c0bf1e013b5ca72` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:19.182207 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cfdd4f-b75c-444f-b094-53d324db4ae0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85b9d4e830684dca9c0bf1e013b5ca72)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:19.184557 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cfdd4f-b75c-444f-b094-53d324db4ae0 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:19.223791 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-f41b34b6-9a9c-4f94-81c0-bf28ba712492 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af0b038d876c43269fb5e518779c2d9c', 'request_id': 'req-f41b34b6-9a9c-4f94-81c0-bf28ba712492'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:19.233064 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-f41b34b6-9a9c-4f94-81c0-bf28ba712492 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af0b038d876c43269fb5e518779c2d9c', 'request_id': 'req-f41b34b6-9a9c-4f94-81c0-bf28ba712492'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:19.245809 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-f41b34b6-9a9c-4f94-81c0-bf28ba712492 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af0b038d876c43269fb5e518779c2d9c', 'request_id': 'req-f41b34b6-9a9c-4f94-81c0-bf28ba712492'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:19.261839 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-f41b34b6-9a9c-4f94-81c0-bf28ba712492 admin admin] The token cache is being invalidate because project af0b038d876c43269fb5e518779c2d9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81712) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:16:19.262420 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-f41b34b6-9a9c-4f94-81c0-bf28ba712492 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:19.263529 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-f41b34b6-9a9c-4f94-81c0-bf28ba712492 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:19.264644 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-14cfdd4f-b75c-444f-b094-53d324db4ae0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85b9d4e830684dca9c0bf1e013b5ca72', 'request_id': 'req-14cfdd4f-b75c-444f-b094-53d324db4ae0'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:19.267966 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1143/2281] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:16:19 2022] DELETE /identity/v3/projects/af0b038d876c43269fb5e518779c2d9c => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:16:19.269878 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7490f62a-d3af-4b84-ba1f-5864e349268d None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:19.270259 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7490f62a-d3af-4b84-ba1f-5864e349268d None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:19.270631 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7490f62a-d3af-4b84-ba1f-5864e349268d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:19.274604 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-14cfdd4f-b75c-444f-b094-53d324db4ae0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85b9d4e830684dca9c0bf1e013b5ca72', 'request_id': 'req-14cfdd4f-b75c-444f-b094-53d324db4ae0'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:19.285624 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-14cfdd4f-b75c-444f-b094-53d324db4ae0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85b9d4e830684dca9c0bf1e013b5ca72', 'request_id': 'req-14cfdd4f-b75c-444f-b094-53d324db4ae0'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:19.299958 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-7490f62a-d3af-4b84-ba1f-5864e349268d None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:16:19.301033 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-14cfdd4f-b75c-444f-b094-53d324db4ae0 admin admin] The token cache is being invalidate because project 85b9d4e830684dca9c0bf1e013b5ca72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:16:19.301159 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-7490f62a-d3af-4b84-ba1f-5864e349268d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:19.301680 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-14cfdd4f-b75c-444f-b094-53d324db4ae0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:19.302576 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-14cfdd4f-b75c-444f-b094-53d324db4ae0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:19.304724 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1139/2282] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:16:19 2022] DELETE /identity/v3/projects/85b9d4e830684dca9c0bf1e013b5ca72 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 12:16:19.306593 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cfdd4f-b75c-444f-b094-53d324db4ae0 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:19.308339 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-14cfdd4f-b75c-444f-b094-53d324db4ae0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:19.399253 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1144/2283] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:16:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:16:19.400607 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7490f62a-d3af-4b84-ba1f-5864e349268d None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:19.416062 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7490f62a-d3af-4b84-ba1f-5864e349268d None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:19.422245 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2526ceda-c9df-467d-8007-d51dbb309e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:19.423470 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2526ceda-c9df-467d-8007-d51dbb309e3c admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:19.424009 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2526ceda-c9df-467d-8007-d51dbb309e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:19.424505 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2526ceda-c9df-467d-8007-d51dbb309e3c admin admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:19.425697 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2526ceda-c9df-467d-8007-d51dbb309e3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:16:19.426110 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2526ceda-c9df-467d-8007-d51dbb309e3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:19.427950 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2526ceda-c9df-467d-8007-d51dbb309e3c admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:19.434153 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cfdd4f-b75c-444f-b094-53d324db4ae0 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:19.447242 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56f9121-6a96-44cd-9672-7c4f9d55381b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:19.447242 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f9121-6a96-44cd-9672-7c4f9d55381b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:19.447242 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f9121-6a96-44cd-9672-7c4f9d55381b admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:19.447242 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f9121-6a96-44cd-9672-7c4f9d55381b admin admin] PATH_INFO: `/v3/users/356867146ad4406caa2e88d58bee8a28` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:19.447706 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56f9121-6a96-44cd-9672-7c4f9d55381b admin admin] RBAC: Authorizing `identity:delete_user(user_id=356867146ad4406caa2e88d58bee8a28)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:19.447706 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56f9121-6a96-44cd-9672-7c4f9d55381b admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:19.447846 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1145/2284] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:16:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:16:19.450025 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2526ceda-c9df-467d-8007-d51dbb309e3c admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:19.461870 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2526ceda-c9df-467d-8007-d51dbb309e3c admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:19.467608 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa7f81-fd91-4757-bead-704c01fef2b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:19.468242 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa7f81-fd91-4757-bead-704c01fef2b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:19.468524 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa7f81-fd91-4757-bead-704c01fef2b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:19.468802 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa7f81-fd91-4757-bead-704c01fef2b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:19.469447 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fa7f81-fd91-4757-bead-704c01fef2b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:19.471425 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fa7f81-fd91-4757-bead-704c01fef2b6 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:19.503954 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1146/2285] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:16:19 2022] POST /identity/v3/projects => generated 399 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 12:16:19.508316 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa7f81-fd91-4757-bead-704c01fef2b6 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:19.515987 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-b56f9121-6a96-44cd-9672-7c4f9d55381b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '356867146ad4406caa2e88d58bee8a28'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:19.525979 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-b56f9121-6a96-44cd-9672-7c4f9d55381b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '356867146ad4406caa2e88d58bee8a28'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:19.528450 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:16:19.535781 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-b56f9121-6a96-44cd-9672-7c4f9d55381b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '356867146ad4406caa2e88d58bee8a28'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:19.537157 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-b56f9121-6a96-44cd-9672-7c4f9d55381b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '356867146ad4406caa2e88d58bee8a28'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:19.551262 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-b56f9121-6a96-44cd-9672-7c4f9d55381b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '356867146ad4406caa2e88d58bee8a28'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:19.557262 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1140/2286] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:16:19 2022] DELETE /identity/v3/users/356867146ad4406caa2e88d58bee8a28 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:16:19.560413 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:16:19.582218 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa7f81-fd91-4757-bead-704c01fef2b6 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:19.641513 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38789770-396e-45cb-9769-fb7d50176300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:19.642123 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38789770-396e-45cb-9769-fb7d50176300 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:19.642399 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38789770-396e-45cb-9769-fb7d50176300 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:19.642665 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38789770-396e-45cb-9769-fb7d50176300 admin admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:19.643698 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38789770-396e-45cb-9769-fb7d50176300 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:19.645244 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38789770-396e-45cb-9769-fb7d50176300 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:19.670752 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1147/2287] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:16:19 2022] POST /identity/v3/users => generated 300 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:16:19.671970 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56f9121-6a96-44cd-9672-7c4f9d55381b admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:19.673389 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-b56f9121-6a96-44cd-9672-7c4f9d55381b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:19.686460 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56f9121-6a96-44cd-9672-7c4f9d55381b admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:19.691223 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921b72f8-0dd7-4514-9daa-7ce2a3a8b999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:19.691829 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b72f8-0dd7-4514-9daa-7ce2a3a8b999 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:19.692104 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b72f8-0dd7-4514-9daa-7ce2a3a8b999 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:19.692374 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b72f8-0dd7-4514-9daa-7ce2a3a8b999 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:19.693006 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921b72f8-0dd7-4514-9daa-7ce2a3a8b999 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:19.694590 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921b72f8-0dd7-4514-9daa-7ce2a3a8b999 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:19.707579 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1141/2288] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:16:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:16:19.708811 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38789770-396e-45cb-9769-fb7d50176300 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:19.722556 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38789770-396e-45cb-9769-fb7d50176300 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:19.728326 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b992224-45ed-4e75-bf2f-f1aec8637dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:19.728956 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b992224-45ed-4e75-bf2f-f1aec8637dc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:19.729234 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b992224-45ed-4e75-bf2f-f1aec8637dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:19.729507 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b992224-45ed-4e75-bf2f-f1aec8637dc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:19.730147 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b992224-45ed-4e75-bf2f-f1aec8637dc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:19.732113 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b992224-45ed-4e75-bf2f-f1aec8637dc7 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:19.746609 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1148/2289] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:16:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:16:19.748165 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921b72f8-0dd7-4514-9daa-7ce2a3a8b999 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:19.758882 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921b72f8-0dd7-4514-9daa-7ce2a3a8b999 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:19.763553 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f49099-0902-45de-8224-a9cf67cf0d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:19.764202 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f49099-0902-45de-8224-a9cf67cf0d40 admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:19.764480 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f49099-0902-45de-8224-a9cf67cf0d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:19.764770 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f49099-0902-45de-8224-a9cf67cf0d40 admin admin] PATH_INFO: `/v3/projects/e05176629598478aa5506490c692a50c/users/a4d7d1ea5fe3473cb9aa9aff7697c203/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:19.780282 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f49099-0902-45de-8224-a9cf67cf0d40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e05176629598478aa5506490c692a50c, user_id=a4d7d1ea5fe3473cb9aa9aff7697c203, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:19.782924 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f49099-0902-45de-8224-a9cf67cf0d40 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:19.796698 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1142/2290] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:16:19 2022] PUT /identity/v3/projects/e05176629598478aa5506490c692a50c/users/a4d7d1ea5fe3473cb9aa9aff7697c203/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:16:19.807006 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605362c1-e15b-4fb3-8f4f-6ff0aac837cd None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:19.807338 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605362c1-e15b-4fb3-8f4f-6ff0aac837cd None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:19.807616 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605362c1-e15b-4fb3-8f4f-6ff0aac837cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:19.839901 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-605362c1-e15b-4fb3-8f4f-6ff0aac837cd None None] MFA Rules not processed for user `a4d7d1ea5fe3473cb9aa9aff7697c203`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:16:19.841385 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-605362c1-e15b-4fb3-8f4f-6ff0aac837cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:19.842335 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f49099-0902-45de-8224-a9cf67cf0d40 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:19.844147 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-11f49099-0902-45de-8224-a9cf67cf0d40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:19.914877 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f49099-0902-45de-8224-a9cf67cf0d40 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:19.917521 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1149/2291] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:16:19 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:16:19.921392 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac6a534-e3b5-4a7f-8ea5-2b291346ced1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:19.922153 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac6a534-e3b5-4a7f-8ea5-2b291346ced1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:19.922434 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac6a534-e3b5-4a7f-8ea5-2b291346ced1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:19.922707 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac6a534-e3b5-4a7f-8ea5-2b291346ced1 admin admin] PATH_INFO: `/v3/projects/6c748e6f935247ac99617e6b1845277b` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:19.924234 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac6a534-e3b5-4a7f-8ea5-2b291346ced1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c748e6f935247ac99617e6b1845277b)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:19.925805 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac6a534-e3b5-4a7f-8ea5-2b291346ced1 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:19.930398 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605362c1-e15b-4fb3-8f4f-6ff0aac837cd None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:19.933098 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-605362c1-e15b-4fb3-8f4f-6ff0aac837cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:19.979483 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-bac6a534-e3b5-4a7f-8ea5-2b291346ced1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c748e6f935247ac99617e6b1845277b', 'request_id': 'req-bac6a534-e3b5-4a7f-8ea5-2b291346ced1'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:19.986449 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-bac6a534-e3b5-4a7f-8ea5-2b291346ced1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c748e6f935247ac99617e6b1845277b', 'request_id': 'req-bac6a534-e3b5-4a7f-8ea5-2b291346ced1'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:19.994443 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605362c1-e15b-4fb3-8f4f-6ff0aac837cd None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:19.996687 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-bac6a534-e3b5-4a7f-8ea5-2b291346ced1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c748e6f935247ac99617e6b1845277b', 'request_id': 'req-bac6a534-e3b5-4a7f-8ea5-2b291346ced1'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:20.005711 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:16:20.010876 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-bac6a534-e3b5-4a7f-8ea5-2b291346ced1 admin admin] The token cache is being invalidate because project 6c748e6f935247ac99617e6b1845277b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:16:20.011016 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-bac6a534-e3b5-4a7f-8ea5-2b291346ced1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:20.012297 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-bac6a534-e3b5-4a7f-8ea5-2b291346ced1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:20.014322 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1143/2292] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:16:19 2022] DELETE /identity/v3/projects/6c748e6f935247ac99617e6b1845277b => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 12:16:20.027445 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc75e0-8b10-4cae-ad05-deb7a9cc4365 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:20.027445 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc75e0-8b10-4cae-ad05-deb7a9cc4365 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:20.027445 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc75e0-8b10-4cae-ad05-deb7a9cc4365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:20.040609 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-c7cc75e0-8b10-4cae-ad05-deb7a9cc4365 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:16:20.041888 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-c7cc75e0-8b10-4cae-ad05-deb7a9cc4365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:20.051548 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efea208-b50c-48f0-9d79-8a5df30bc6e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:20.052083 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efea208-b50c-48f0-9d79-8a5df30bc6e9 service cinder] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:20.052358 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efea208-b50c-48f0-9d79-8a5df30bc6e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:20.052624 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efea208-b50c-48f0-9d79-8a5df30bc6e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:20.054581 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-8efea208-b50c-48f0-9d79-8a5df30bc6e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:20.065781 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:16:20.102987 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1144/2293] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:16:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:16:20.110508 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ac233b-7a10-4df7-8aaf-9e520c4e04b0 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:20.110793 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ac233b-7a10-4df7-8aaf-9e520c4e04b0 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:20.111070 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ac233b-7a10-4df7-8aaf-9e520c4e04b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:20.118981 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efea208-b50c-48f0-9d79-8a5df30bc6e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:20.120496 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efea208-b50c-48f0-9d79-8a5df30bc6e9 service cinder] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:20.130272 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1150/2294] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:16:19 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 12:16:20.131824 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-c7ac233b-7a10-4df7-8aaf-9e520c4e04b0 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:16:20.133795 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-c7ac233b-7a10-4df7-8aaf-9e520c4e04b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:20.143173 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1145/2295] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:16:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:16:20.147515 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efea208-b50c-48f0-9d79-8a5df30bc6e9 service cinder] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:20.158414 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efea208-b50c-48f0-9d79-8a5df30bc6e9 service cinder] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:20.162983 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8452a3d5-72cc-40de-b4b4-fe773f247489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:20.163615 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8452a3d5-72cc-40de-b4b4-fe773f247489 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:20.163894 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8452a3d5-72cc-40de-b4b4-fe773f247489 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:20.164186 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8452a3d5-72cc-40de-b4b4-fe773f247489 admin admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:20.165058 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8452a3d5-72cc-40de-b4b4-fe773f247489 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:16:20.165357 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8452a3d5-72cc-40de-b4b4-fe773f247489 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:20.166936 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8452a3d5-72cc-40de-b4b4-fe773f247489 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:20.178983 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1151/2296] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:16:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:16:20.184107 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ac233b-7a10-4df7-8aaf-9e520c4e04b0 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:20.193323 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ac233b-7a10-4df7-8aaf-9e520c4e04b0 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:20.198895 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f978f679-4a3a-406c-9890-0256cb262eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:20.199520 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f978f679-4a3a-406c-9890-0256cb262eab admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:20.199803 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f978f679-4a3a-406c-9890-0256cb262eab admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:20.200075 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f978f679-4a3a-406c-9890-0256cb262eab admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:20.200705 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f978f679-4a3a-406c-9890-0256cb262eab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:20.202589 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f978f679-4a3a-406c-9890-0256cb262eab admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:20.234363 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1146/2297] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:16:20 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:16:20.240594 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8452a3d5-72cc-40de-b4b4-fe773f247489 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:20.252064 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8452a3d5-72cc-40de-b4b4-fe773f247489 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:20.323942 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a303f7be-6392-493a-ae6d-04f5d3fa3f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:20.324563 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303f7be-6392-493a-ae6d-04f5d3fa3f7c admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:20.324828 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303f7be-6392-493a-ae6d-04f5d3fa3f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:20.325104 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303f7be-6392-493a-ae6d-04f5d3fa3f7c admin admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:20.325727 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a303f7be-6392-493a-ae6d-04f5d3fa3f7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:20.327664 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a303f7be-6392-493a-ae6d-04f5d3fa3f7c admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:20.349309 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1152/2298] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:16:20 2022] POST /identity/v3/users => generated 292 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:16:20.353652 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f978f679-4a3a-406c-9890-0256cb262eab admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:20.364300 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f978f679-4a3a-406c-9890-0256cb262eab admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:20.370409 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b79030e-bc12-4021-99ea-8611e3af6bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:20.371083 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b79030e-bc12-4021-99ea-8611e3af6bbf admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:20.371442 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b79030e-bc12-4021-99ea-8611e3af6bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:20.371810 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b79030e-bc12-4021-99ea-8611e3af6bbf admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:20.372622 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b79030e-bc12-4021-99ea-8611e3af6bbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:20.374223 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b79030e-bc12-4021-99ea-8611e3af6bbf admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:20.385444 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1147/2299] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:16:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 12:16:20.389614 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a303f7be-6392-493a-ae6d-04f5d3fa3f7c admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:20.402321 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a303f7be-6392-493a-ae6d-04f5d3fa3f7c admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:20.407130 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdf160f-1be7-4b5f-8a6c-ad90f40369ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:20.407791 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf160f-1be7-4b5f-8a6c-ad90f40369ea admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:20.408121 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf160f-1be7-4b5f-8a6c-ad90f40369ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:20.408455 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf160f-1be7-4b5f-8a6c-ad90f40369ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:20.409099 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdf160f-1be7-4b5f-8a6c-ad90f40369ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:20.410713 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdf160f-1be7-4b5f-8a6c-ad90f40369ea admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:20.422079 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1153/2300] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:16:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:16:20.426449 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b79030e-bc12-4021-99ea-8611e3af6bbf admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:20.437624 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b79030e-bc12-4021-99ea-8611e3af6bbf admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:20.442321 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925e8ac8-e08a-4862-9266-8116160276ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:20.442974 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925e8ac8-e08a-4862-9266-8116160276ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:20.443277 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925e8ac8-e08a-4862-9266-8116160276ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:20.443554 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925e8ac8-e08a-4862-9266-8116160276ea admin admin] PATH_INFO: `/v3/projects/a135e5b3ed2845eb8fdc41a029254507/users/5f3e7aea86c44e21ad670d71835de6de/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:20.460505 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925e8ac8-e08a-4862-9266-8116160276ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=a135e5b3ed2845eb8fdc41a029254507, user_id=5f3e7aea86c44e21ad670d71835de6de, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:20.463451 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925e8ac8-e08a-4862-9266-8116160276ea admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:20.478277 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1148/2301] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:16:20 2022] PUT /identity/v3/projects/a135e5b3ed2845eb8fdc41a029254507/users/5f3e7aea86c44e21ad670d71835de6de/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 12:16:20.492150 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34107cc-1925-44d2-a187-02d58afe3d09 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:20.493212 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34107cc-1925-44d2-a187-02d58afe3d09 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:20.493343 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34107cc-1925-44d2-a187-02d58afe3d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:20.543409 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-e34107cc-1925-44d2-a187-02d58afe3d09 None None] MFA Rules not processed for user `5f3e7aea86c44e21ad670d71835de6de`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:16:20.545180 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-e34107cc-1925-44d2-a187-02d58afe3d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:20.569010 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:16:20.641937 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1154/2302] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:16:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:16:20.643161 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925e8ac8-e08a-4862-9266-8116160276ea admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:20.716745 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925e8ac8-e08a-4862-9266-8116160276ea admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:20.721345 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dd02dd-15df-4d89-86a3-80e0f3cb552a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:20.721941 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd02dd-15df-4d89-86a3-80e0f3cb552a admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:20.722220 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd02dd-15df-4d89-86a3-80e0f3cb552a admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:20.722506 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd02dd-15df-4d89-86a3-80e0f3cb552a admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:20.723107 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dd02dd-15df-4d89-86a3-80e0f3cb552a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:20.725049 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dd02dd-15df-4d89-86a3-80e0f3cb552a admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:20.756693 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1149/2303] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:16:20 2022] POST /identity/v3/projects => generated 383 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:16:20.760904 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34107cc-1925-44d2-a187-02d58afe3d09 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:20.771529 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34107cc-1925-44d2-a187-02d58afe3d09 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:20.828038 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eeb6d3-f6c7-4e8f-838d-db35209b8084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:20.828557 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eeb6d3-f6c7-4e8f-838d-db35209b8084 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:20.828834 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eeb6d3-f6c7-4e8f-838d-db35209b8084 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:20.829100 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eeb6d3-f6c7-4e8f-838d-db35209b8084 admin admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:20.829719 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eeb6d3-f6c7-4e8f-838d-db35209b8084 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:20.831652 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eeb6d3-f6c7-4e8f-838d-db35209b8084 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:20.853818 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1155/2304] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:16:20 2022] POST /identity/v3/users => generated 298 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:16:20.857895 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dd02dd-15df-4d89-86a3-80e0f3cb552a admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:20.869091 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dd02dd-15df-4d89-86a3-80e0f3cb552a admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:20.873966 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425b402f-2493-4202-b31c-2e56a93aa4eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:20.874598 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b402f-2493-4202-b31c-2e56a93aa4eb admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:20.874879 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b402f-2493-4202-b31c-2e56a93aa4eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:20.875307 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b402f-2493-4202-b31c-2e56a93aa4eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:20.875949 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425b402f-2493-4202-b31c-2e56a93aa4eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:20.877538 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425b402f-2493-4202-b31c-2e56a93aa4eb admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:20.888067 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1150/2305] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:16:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:16:20.892157 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eeb6d3-f6c7-4e8f-838d-db35209b8084 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:20.902682 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eeb6d3-f6c7-4e8f-838d-db35209b8084 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:20.907469 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29326b9d-a933-4690-8c1f-579adaf27210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:20.908113 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29326b9d-a933-4690-8c1f-579adaf27210 admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:20.908393 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29326b9d-a933-4690-8c1f-579adaf27210 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:20.908663 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29326b9d-a933-4690-8c1f-579adaf27210 admin admin] PATH_INFO: `/v3/projects/c550d2bd788c44fb91c4b30d7b00c33b/users/e6a36b0b1b1d4995bcad33f3ad0cfcb7/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:20.923919 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29326b9d-a933-4690-8c1f-579adaf27210 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c550d2bd788c44fb91c4b30d7b00c33b, user_id=e6a36b0b1b1d4995bcad33f3ad0cfcb7, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:20.926582 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29326b9d-a933-4690-8c1f-579adaf27210 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:20.939094 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1156/2306] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:16:20 2022] PUT /identity/v3/projects/c550d2bd788c44fb91c4b30d7b00c33b/users/e6a36b0b1b1d4995bcad33f3ad0cfcb7/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:16:20.943050 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425b402f-2493-4202-b31c-2e56a93aa4eb admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:20.952986 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425b402f-2493-4202-b31c-2e56a93aa4eb admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:20.990856 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:16:21.010138 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5dc0d1-3a30-4c45-93a3-8abd3ffa9cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:21.010747 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5dc0d1-3a30-4c45-93a3-8abd3ffa9cc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:21.011026 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5dc0d1-3a30-4c45-93a3-8abd3ffa9cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:21.011326 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5dc0d1-3a30-4c45-93a3-8abd3ffa9cc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:21.011956 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5dc0d1-3a30-4c45-93a3-8abd3ffa9cc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:21.013587 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5dc0d1-3a30-4c45-93a3-8abd3ffa9cc1 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:21.023227 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1151/2307] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:16:20 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 12:16:21.027860 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29326b9d-a933-4690-8c1f-579adaf27210 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:21.038394 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29326b9d-a933-4690-8c1f-579adaf27210 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:21.043946 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ef3faa-b97b-4512-b804-c4d181efbbf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:21.044597 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ef3faa-b97b-4512-b804-c4d181efbbf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:21.044878 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ef3faa-b97b-4512-b804-c4d181efbbf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:21.045156 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ef3faa-b97b-4512-b804-c4d181efbbf2 admin admin] PATH_INFO: `/v3/projects/c550d2bd788c44fb91c4b30d7b00c33b/users/e6a36b0b1b1d4995bcad33f3ad0cfcb7/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:21.047071 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ef3faa-b97b-4512-b804-c4d181efbbf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c550d2bd788c44fb91c4b30d7b00c33b, user_id=e6a36b0b1b1d4995bcad33f3ad0cfcb7, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:21.049745 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ef3faa-b97b-4512-b804-c4d181efbbf2 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:21.060706 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1157/2308] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:16:21 2022] PUT /identity/v3/projects/c550d2bd788c44fb91c4b30d7b00c33b/users/e6a36b0b1b1d4995bcad33f3ad0cfcb7/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:16:21.071059 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b55972-17e5-41f2-90f0-a3efbe099cec None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:21.071383 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b55972-17e5-41f2-90f0-a3efbe099cec None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:21.071659 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b55972-17e5-41f2-90f0-a3efbe099cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:21.100823 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-87b55972-17e5-41f2-90f0-a3efbe099cec None None] MFA Rules not processed for user `e6a36b0b1b1d4995bcad33f3ad0cfcb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:16:21.102656 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-87b55972-17e5-41f2-90f0-a3efbe099cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:21.170655 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1152/2309] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:16:21 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:16:21.182316 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ef3faa-b97b-4512-b804-c4d181efbbf2 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:21.184026 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-07ef3faa-b97b-4512-b804-c4d181efbbf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:21.241433 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ef3faa-b97b-4512-b804-c4d181efbbf2 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:21.245257 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c70c335-b5ef-4725-8d91-bf1cba92c920 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:21.245853 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c70c335-b5ef-4725-8d91-bf1cba92c920 service cinder] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:21.246130 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c70c335-b5ef-4725-8d91-bf1cba92c920 service cinder] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:21.246402 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c70c335-b5ef-4725-8d91-bf1cba92c920 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:21.255085 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c70c335-b5ef-4725-8d91-bf1cba92c920 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:21.256529 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c70c335-b5ef-4725-8d91-bf1cba92c920 service cinder] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:21.266460 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1158/2310] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:16:21 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:16:21.340721 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b55972-17e5-41f2-90f0-a3efbe099cec None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:21.342154 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-87b55972-17e5-41f2-90f0-a3efbe099cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:21.353689 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b55972-17e5-41f2-90f0-a3efbe099cec None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:21.357269 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4884f24-c5f1-4b2b-9a75-52db551d5211 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:21.357861 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4884f24-c5f1-4b2b-9a75-52db551d5211 service cinder] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:21.358138 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4884f24-c5f1-4b2b-9a75-52db551d5211 service cinder] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:21.358411 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4884f24-c5f1-4b2b-9a75-52db551d5211 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:21.414936 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4884f24-c5f1-4b2b-9a75-52db551d5211 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:21.416385 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4884f24-c5f1-4b2b-9a75-52db551d5211 service cinder] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:21.424997 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1153/2311] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:16:21 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:16:21.690478 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c70c335-b5ef-4725-8d91-bf1cba92c920 service cinder] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:21.708899 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:16:21.766283 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c70c335-b5ef-4725-8d91-bf1cba92c920 service cinder] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:21.771178 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c0b26e-3aae-4c06-af9d-ed11206a3b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:21.771908 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c0b26e-3aae-4c06-af9d-ed11206a3b61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:21.772250 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c0b26e-3aae-4c06-af9d-ed11206a3b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:21.772583 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c0b26e-3aae-4c06-af9d-ed11206a3b61 admin admin] PATH_INFO: `/v3/users/5f3e7aea86c44e21ad670d71835de6de` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:21.773816 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c0b26e-3aae-4c06-af9d-ed11206a3b61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f3e7aea86c44e21ad670d71835de6de)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:21.775954 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c0b26e-3aae-4c06-af9d-ed11206a3b61 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:21.824781 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-40c0b26e-3aae-4c06-af9d-ed11206a3b61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f3e7aea86c44e21ad670d71835de6de'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:21.836622 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-40c0b26e-3aae-4c06-af9d-ed11206a3b61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f3e7aea86c44e21ad670d71835de6de'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:21.837311 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-40c0b26e-3aae-4c06-af9d-ed11206a3b61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f3e7aea86c44e21ad670d71835de6de'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:21.844560 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-40c0b26e-3aae-4c06-af9d-ed11206a3b61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f3e7aea86c44e21ad670d71835de6de'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:21.845200 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-40c0b26e-3aae-4c06-af9d-ed11206a3b61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f3e7aea86c44e21ad670d71835de6de'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:21.862256 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1159/2312] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:16:21 2022] DELETE /identity/v3/users/5f3e7aea86c44e21ad670d71835de6de => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 12:16:21.866926 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f8a71-c2ad-4e88-b534-7f2aaeee1758 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:21.867974 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f8a71-c2ad-4e88-b534-7f2aaeee1758 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:21.868259 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f8a71-c2ad-4e88-b534-7f2aaeee1758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:21.888778 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-3f9f8a71-c2ad-4e88-b534-7f2aaeee1758 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:16:21.889946 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-3f9f8a71-c2ad-4e88-b534-7f2aaeee1758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:21.938639 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:16:21.947695 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1154/2313] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:16:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:16:21.957409 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c0b26e-3aae-4c06-af9d-ed11206a3b61 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:21.958907 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-40c0b26e-3aae-4c06-af9d-ed11206a3b61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:22.015931 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c0b26e-3aae-4c06-af9d-ed11206a3b61 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:22.019505 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b921ccad-5b55-4368-88c0-0bb0586b8e83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:22.020100 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921ccad-5b55-4368-88c0-0bb0586b8e83 service neutron] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:22.020375 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921ccad-5b55-4368-88c0-0bb0586b8e83 service neutron] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:22.020644 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921ccad-5b55-4368-88c0-0bb0586b8e83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:22.029637 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b921ccad-5b55-4368-88c0-0bb0586b8e83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:22.031111 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b921ccad-5b55-4368-88c0-0bb0586b8e83 service neutron] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:22.041367 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1160/2314] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:16:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:16:22.318743 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9f8a71-c2ad-4e88-b534-7f2aaeee1758 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:22.320251 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-3f9f8a71-c2ad-4e88-b534-7f2aaeee1758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:22.332426 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9f8a71-c2ad-4e88-b534-7f2aaeee1758 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:22.338556 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd60ab6-78b7-478f-a9e7-7213231f8a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:22.339348 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd60ab6-78b7-478f-a9e7-7213231f8a26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:22.339625 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd60ab6-78b7-478f-a9e7-7213231f8a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:22.340233 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd60ab6-78b7-478f-a9e7-7213231f8a26 admin admin] PATH_INFO: `/v3/projects/a135e5b3ed2845eb8fdc41a029254507` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:22.341014 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd60ab6-78b7-478f-a9e7-7213231f8a26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a135e5b3ed2845eb8fdc41a029254507)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:22.342901 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd60ab6-78b7-478f-a9e7-7213231f8a26 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:22.396443 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-fdd60ab6-78b7-478f-a9e7-7213231f8a26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a135e5b3ed2845eb8fdc41a029254507', 'request_id': 'req-fdd60ab6-78b7-478f-a9e7-7213231f8a26'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:22.403036 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-fdd60ab6-78b7-478f-a9e7-7213231f8a26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a135e5b3ed2845eb8fdc41a029254507', 'request_id': 'req-fdd60ab6-78b7-478f-a9e7-7213231f8a26'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:22.410324 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-fdd60ab6-78b7-478f-a9e7-7213231f8a26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a135e5b3ed2845eb8fdc41a029254507', 'request_id': 'req-fdd60ab6-78b7-478f-a9e7-7213231f8a26'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:22.421160 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-fdd60ab6-78b7-478f-a9e7-7213231f8a26 admin admin] The token cache is being invalidate because project a135e5b3ed2845eb8fdc41a029254507 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:16:22.421445 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-fdd60ab6-78b7-478f-a9e7-7213231f8a26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:22.422111 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-fdd60ab6-78b7-478f-a9e7-7213231f8a26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:22.423610 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1155/2315] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:16:22 2022] DELETE /identity/v3/projects/a135e5b3ed2845eb8fdc41a029254507 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:16:22.427680 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b921ccad-5b55-4368-88c0-0bb0586b8e83 service neutron] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:22.437773 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b921ccad-5b55-4368-88c0-0bb0586b8e83 service neutron] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:22.471645 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:16:22.499072 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92de66ce-d9a7-4822-904c-7b21d06b683e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:22.499733 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de66ce-d9a7-4822-904c-7b21d06b683e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:22.500048 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de66ce-d9a7-4822-904c-7b21d06b683e admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:22.500326 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de66ce-d9a7-4822-904c-7b21d06b683e admin admin] PATH_INFO: `/v3/users/e6a36b0b1b1d4995bcad33f3ad0cfcb7` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:22.501472 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92de66ce-d9a7-4822-904c-7b21d06b683e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6a36b0b1b1d4995bcad33f3ad0cfcb7)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:22.503467 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92de66ce-d9a7-4822-904c-7b21d06b683e admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:22.550254 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-92de66ce-d9a7-4822-904c-7b21d06b683e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6a36b0b1b1d4995bcad33f3ad0cfcb7'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:22.557344 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-92de66ce-d9a7-4822-904c-7b21d06b683e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6a36b0b1b1d4995bcad33f3ad0cfcb7'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:22.558040 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-92de66ce-d9a7-4822-904c-7b21d06b683e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6a36b0b1b1d4995bcad33f3ad0cfcb7'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:22.566362 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-92de66ce-d9a7-4822-904c-7b21d06b683e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6a36b0b1b1d4995bcad33f3ad0cfcb7'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:22.567069 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-92de66ce-d9a7-4822-904c-7b21d06b683e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6a36b0b1b1d4995bcad33f3ad0cfcb7'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:22.585202 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1161/2316] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:16:22 2022] DELETE /identity/v3/users/e6a36b0b1b1d4995bcad33f3ad0cfcb7 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:16:22.897787 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd60ab6-78b7-478f-a9e7-7213231f8a26 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:22.899579 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-fdd60ab6-78b7-478f-a9e7-7213231f8a26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:22.920466 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd60ab6-78b7-478f-a9e7-7213231f8a26 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:22.987370 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c430151d-1d32-44f9-a4cb-20823dea22ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:22.988057 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c430151d-1d32-44f9-a4cb-20823dea22ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:22.988663 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c430151d-1d32-44f9-a4cb-20823dea22ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:22.988956 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c430151d-1d32-44f9-a4cb-20823dea22ef admin admin] PATH_INFO: `/v3/projects/c550d2bd788c44fb91c4b30d7b00c33b` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:22.990156 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c430151d-1d32-44f9-a4cb-20823dea22ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=c550d2bd788c44fb91c4b30d7b00c33b)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:22.993069 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c430151d-1d32-44f9-a4cb-20823dea22ef admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:23.044788 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-c430151d-1d32-44f9-a4cb-20823dea22ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c550d2bd788c44fb91c4b30d7b00c33b', 'request_id': 'req-c430151d-1d32-44f9-a4cb-20823dea22ef'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:23.051093 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-c430151d-1d32-44f9-a4cb-20823dea22ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c550d2bd788c44fb91c4b30d7b00c33b', 'request_id': 'req-c430151d-1d32-44f9-a4cb-20823dea22ef'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:23.059301 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-c430151d-1d32-44f9-a4cb-20823dea22ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c550d2bd788c44fb91c4b30d7b00c33b', 'request_id': 'req-c430151d-1d32-44f9-a4cb-20823dea22ef'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:23.070193 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-c430151d-1d32-44f9-a4cb-20823dea22ef admin admin] The token cache is being invalidate because project c550d2bd788c44fb91c4b30d7b00c33b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:16:23.070503 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-c430151d-1d32-44f9-a4cb-20823dea22ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:23.071234 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-c430151d-1d32-44f9-a4cb-20823dea22ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:23.072721 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1156/2317] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:16:22 2022] DELETE /identity/v3/projects/c550d2bd788c44fb91c4b30d7b00c33b => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:16:23.077513 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8e6a33-0f2c-4f60-8693-7fa1660bb5ac None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:23.078081 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8e6a33-0f2c-4f60-8693-7fa1660bb5ac None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:23.078081 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8e6a33-0f2c-4f60-8693-7fa1660bb5ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:23.097245 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-ed8e6a33-0f2c-4f60-8693-7fa1660bb5ac None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:16:23.098441 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-ed8e6a33-0f2c-4f60-8693-7fa1660bb5ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:23.158573 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1162/2318] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:16:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:16:23.166908 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cea78f-121a-4dfc-8f49-33d062f82399 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:23.167264 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cea78f-121a-4dfc-8f49-33d062f82399 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:23.167546 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cea78f-121a-4dfc-8f49-33d062f82399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:23.187333 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-d9cea78f-121a-4dfc-8f49-33d062f82399 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:16:23.188539 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-d9cea78f-121a-4dfc-8f49-33d062f82399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:23.196581 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1157/2319] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:16:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:16:23.200968 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8e6a33-0f2c-4f60-8693-7fa1660bb5ac None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:23.211683 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8e6a33-0f2c-4f60-8693-7fa1660bb5ac None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:23.216724 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f603a9-49de-4ccb-bab4-351250e8af17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:23.217352 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f603a9-49de-4ccb-bab4-351250e8af17 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:23.217634 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f603a9-49de-4ccb-bab4-351250e8af17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:23.217910 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f603a9-49de-4ccb-bab4-351250e8af17 admin admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:23.218559 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f603a9-49de-4ccb-bab4-351250e8af17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:16:23.218864 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f603a9-49de-4ccb-bab4-351250e8af17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:23.220491 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f603a9-49de-4ccb-bab4-351250e8af17 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:23.237544 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1163/2320] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:16:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:16:23.238938 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cea78f-121a-4dfc-8f49-33d062f82399 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:23.248687 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cea78f-121a-4dfc-8f49-33d062f82399 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:23.253306 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2d2c5e-6685-45aa-9b51-18a991ddc5e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:23.253916 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d2c5e-6685-45aa-9b51-18a991ddc5e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:23.254192 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d2c5e-6685-45aa-9b51-18a991ddc5e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:23.254458 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d2c5e-6685-45aa-9b51-18a991ddc5e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:23.255065 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2d2c5e-6685-45aa-9b51-18a991ddc5e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:23.256977 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2d2c5e-6685-45aa-9b51-18a991ddc5e2 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:23.261368 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:16:23.286330 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1158/2321] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:16:23 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:16:23.291429 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f603a9-49de-4ccb-bab4-351250e8af17 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:23.301176 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f603a9-49de-4ccb-bab4-351250e8af17 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:23.341511 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:16:23.357737 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba4d3bd-ef89-4fe3-9f34-ce725e97b96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:23.358334 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba4d3bd-ef89-4fe3-9f34-ce725e97b96f admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:23.358612 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba4d3bd-ef89-4fe3-9f34-ce725e97b96f admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:23.358884 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba4d3bd-ef89-4fe3-9f34-ce725e97b96f admin admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:23.359551 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba4d3bd-ef89-4fe3-9f34-ce725e97b96f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:23.361498 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba4d3bd-ef89-4fe3-9f34-ce725e97b96f admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:23.382044 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1164/2322] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:16:23 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:16:23.386808 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2d2c5e-6685-45aa-9b51-18a991ddc5e2 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:23.397071 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2d2c5e-6685-45aa-9b51-18a991ddc5e2 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:23.401593 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7659001-7713-4fdc-abc9-6f36fa63dbe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:23.402190 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7659001-7713-4fdc-abc9-6f36fa63dbe0 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:23.402471 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7659001-7713-4fdc-abc9-6f36fa63dbe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:23.402741 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7659001-7713-4fdc-abc9-6f36fa63dbe0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:23.403407 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7659001-7713-4fdc-abc9-6f36fa63dbe0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:23.405877 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7659001-7713-4fdc-abc9-6f36fa63dbe0 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:23.416537 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1159/2323] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:16:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:16:23.421006 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba4d3bd-ef89-4fe3-9f34-ce725e97b96f admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:23.432410 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba4d3bd-ef89-4fe3-9f34-ce725e97b96f admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:23.439505 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32fc86f-194b-40ec-ac85-6d54edd6a745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:23.440108 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32fc86f-194b-40ec-ac85-6d54edd6a745 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:23.440388 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32fc86f-194b-40ec-ac85-6d54edd6a745 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:23.440662 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32fc86f-194b-40ec-ac85-6d54edd6a745 admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:23.441301 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32fc86f-194b-40ec-ac85-6d54edd6a745 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:23.442934 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32fc86f-194b-40ec-ac85-6d54edd6a745 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:23.454314 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1165/2324] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:16:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:16:23.458295 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7659001-7713-4fdc-abc9-6f36fa63dbe0 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:23.468224 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7659001-7713-4fdc-abc9-6f36fa63dbe0 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:23.472848 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7d258a-3cfc-4697-b575-0076fe51f265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:23.473492 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d258a-3cfc-4697-b575-0076fe51f265 admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:23.473774 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d258a-3cfc-4697-b575-0076fe51f265 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:23.474049 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d258a-3cfc-4697-b575-0076fe51f265 admin admin] PATH_INFO: `/v3/projects/501f5b7617494beba3e69610feb4ee8c/users/51568d6bbe0b4dd0ac7be5029f1aaeae/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:23.491231 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7d258a-3cfc-4697-b575-0076fe51f265 admin admin] RBAC: Authorizing `identity:create_grant(project_id=501f5b7617494beba3e69610feb4ee8c, user_id=51568d6bbe0b4dd0ac7be5029f1aaeae, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:23.495127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7d258a-3cfc-4697-b575-0076fe51f265 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:23.511089 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1160/2325] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:16:23 2022] PUT /identity/v3/projects/501f5b7617494beba3e69610feb4ee8c/users/51568d6bbe0b4dd0ac7be5029f1aaeae/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 12:16:23.522634 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a719b-4e0b-4d3b-840b-26f59ccedf96 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:23.522928 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a719b-4e0b-4d3b-840b-26f59ccedf96 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:23.523238 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a719b-4e0b-4d3b-840b-26f59ccedf96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:23.551029 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-150a719b-4e0b-4d3b-840b-26f59ccedf96 None None] MFA Rules not processed for user `51568d6bbe0b4dd0ac7be5029f1aaeae`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:16:23.552254 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-150a719b-4e0b-4d3b-840b-26f59ccedf96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:23.615620 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1166/2326] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:16:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:16:23.627639 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7d258a-3cfc-4697-b575-0076fe51f265 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:23.629553 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-ab7d258a-3cfc-4697-b575-0076fe51f265 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:23.686723 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7d258a-3cfc-4697-b575-0076fe51f265 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:23.690288 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79e9388-147e-4f03-9142-ee88bfd7cf59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:23.690884 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e9388-147e-4f03-9142-ee88bfd7cf59 service cinder] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:23.691161 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e9388-147e-4f03-9142-ee88bfd7cf59 service cinder] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:23.691465 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e9388-147e-4f03-9142-ee88bfd7cf59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:23.699654 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79e9388-147e-4f03-9142-ee88bfd7cf59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:23.701075 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79e9388-147e-4f03-9142-ee88bfd7cf59 service cinder] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:23.709606 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1161/2327] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:16:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 12:16:23.730055 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150a719b-4e0b-4d3b-840b-26f59ccedf96 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:23.795582 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150a719b-4e0b-4d3b-840b-26f59ccedf96 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:23.800576 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d93c28b-e003-4fd6-a4c1-61dd73275051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:23.801236 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93c28b-e003-4fd6-a4c1-61dd73275051 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:23.801515 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93c28b-e003-4fd6-a4c1-61dd73275051 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:23.801789 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93c28b-e003-4fd6-a4c1-61dd73275051 admin admin] PATH_INFO: `/v3/users/51568d6bbe0b4dd0ac7be5029f1aaeae` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:23.802951 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d93c28b-e003-4fd6-a4c1-61dd73275051 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51568d6bbe0b4dd0ac7be5029f1aaeae)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:23.805651 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d93c28b-e003-4fd6-a4c1-61dd73275051 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:23.857557 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-4d93c28b-e003-4fd6-a4c1-61dd73275051 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51568d6bbe0b4dd0ac7be5029f1aaeae'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:23.865537 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-4d93c28b-e003-4fd6-a4c1-61dd73275051 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51568d6bbe0b4dd0ac7be5029f1aaeae'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:23.866417 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-4d93c28b-e003-4fd6-a4c1-61dd73275051 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51568d6bbe0b4dd0ac7be5029f1aaeae'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:23.868062 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:16:23.880952 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-4d93c28b-e003-4fd6-a4c1-61dd73275051 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51568d6bbe0b4dd0ac7be5029f1aaeae'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:23.881670 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-4d93c28b-e003-4fd6-a4c1-61dd73275051 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51568d6bbe0b4dd0ac7be5029f1aaeae'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:23.900684 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1167/2328] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:16:23 2022] DELETE /identity/v3/users/51568d6bbe0b4dd0ac7be5029f1aaeae => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:16:23.905350 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931691bc-0f6e-4ba5-b2dc-b593b72bfa08 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:23.905988 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931691bc-0f6e-4ba5-b2dc-b593b72bfa08 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:23.906097 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931691bc-0f6e-4ba5-b2dc-b593b72bfa08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:23.928559 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-931691bc-0f6e-4ba5-b2dc-b593b72bfa08 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:16:23.930202 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-931691bc-0f6e-4ba5-b2dc-b593b72bfa08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:23.997038 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1162/2329] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:16:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:16:24.006619 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d93c28b-e003-4fd6-a4c1-61dd73275051 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:24.008258 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-4d93c28b-e003-4fd6-a4c1-61dd73275051 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:24.064724 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d93c28b-e003-4fd6-a4c1-61dd73275051 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:24.068424 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff77caac-eed1-4369-850b-728f5efce429 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:24.069018 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff77caac-eed1-4369-850b-728f5efce429 service neutron] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:24.069293 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff77caac-eed1-4369-850b-728f5efce429 service neutron] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:24.069561 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff77caac-eed1-4369-850b-728f5efce429 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:24.078856 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff77caac-eed1-4369-850b-728f5efce429 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:24.080381 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff77caac-eed1-4369-850b-728f5efce429 service neutron] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:24.090385 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1168/2330] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:16:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:16:24.382832 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0311c0aa-f079-4770-9d49-c5650b079567 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:24.383685 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0311c0aa-f079-4770-9d49-c5650b079567 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:24.384044 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0311c0aa-f079-4770-9d49-c5650b079567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:24.404253 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-0311c0aa-f079-4770-9d49-c5650b079567 None None] MFA Rules not processed for user `2b51cd55973a419cad6311609395bb89`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:16:24.405484 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-0311c0aa-f079-4770-9d49-c5650b079567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:24.458046 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:16:24.468854 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1163/2331] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:16:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:16:24.663562 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff77caac-eed1-4369-850b-728f5efce429 service neutron] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:24.665347 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-ff77caac-eed1-4369-850b-728f5efce429 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:24.686851 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff77caac-eed1-4369-850b-728f5efce429 service neutron] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:24.691814 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b2d45e-b277-49ee-9f3e-ff1331f0a02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:24.692379 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b2d45e-b277-49ee-9f3e-ff1331f0a02b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:24.692663 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b2d45e-b277-49ee-9f3e-ff1331f0a02b admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:24.692937 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b2d45e-b277-49ee-9f3e-ff1331f0a02b admin admin] PATH_INFO: `/v3/projects/501f5b7617494beba3e69610feb4ee8c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:24.702757 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b2d45e-b277-49ee-9f3e-ff1331f0a02b admin admin] RBAC: Authorizing `identity:delete_project(project_id=501f5b7617494beba3e69610feb4ee8c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:24.704780 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b2d45e-b277-49ee-9f3e-ff1331f0a02b admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:24.755024 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-13b2d45e-b277-49ee-9f3e-ff1331f0a02b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '501f5b7617494beba3e69610feb4ee8c', 'request_id': 'req-13b2d45e-b277-49ee-9f3e-ff1331f0a02b'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:24.762095 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-13b2d45e-b277-49ee-9f3e-ff1331f0a02b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '501f5b7617494beba3e69610feb4ee8c', 'request_id': 'req-13b2d45e-b277-49ee-9f3e-ff1331f0a02b'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:24.770280 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-13b2d45e-b277-49ee-9f3e-ff1331f0a02b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '501f5b7617494beba3e69610feb4ee8c', 'request_id': 'req-13b2d45e-b277-49ee-9f3e-ff1331f0a02b'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:24.782201 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-13b2d45e-b277-49ee-9f3e-ff1331f0a02b admin admin] The token cache is being invalidate because project 501f5b7617494beba3e69610feb4ee8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81712) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:16:24.782491 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-13b2d45e-b277-49ee-9f3e-ff1331f0a02b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:24.783137 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-13b2d45e-b277-49ee-9f3e-ff1331f0a02b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:24.784644 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1169/2332] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:16:24 2022] DELETE /identity/v3/projects/501f5b7617494beba3e69610feb4ee8c => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:16:24.789390 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779d1a73-5413-4405-9902-514975d44920 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:24.789678 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779d1a73-5413-4405-9902-514975d44920 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:24.789946 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779d1a73-5413-4405-9902-514975d44920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:24.810097 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-779d1a73-5413-4405-9902-514975d44920 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:16:24.811338 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-779d1a73-5413-4405-9902-514975d44920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:24.871970 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1164/2333] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:16:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:16:24.879565 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7bf2ac-f4e0-4372-9ee1-641ad9191bcc None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:24.879850 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7bf2ac-f4e0-4372-9ee1-641ad9191bcc None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:24.880125 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7bf2ac-f4e0-4372-9ee1-641ad9191bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:24.900319 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-6e7bf2ac-f4e0-4372-9ee1-641ad9191bcc None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:16:24.901526 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-6e7bf2ac-f4e0-4372-9ee1-641ad9191bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:24.909690 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1170/2334] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:16:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:16:24.913910 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779d1a73-5413-4405-9902-514975d44920 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:24.924792 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779d1a73-5413-4405-9902-514975d44920 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:24.929634 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29941d3f-fec5-4727-a784-5a56351a94a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:24.930923 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29941d3f-fec5-4727-a784-5a56351a94a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:24.931365 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29941d3f-fec5-4727-a784-5a56351a94a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:24.931721 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29941d3f-fec5-4727-a784-5a56351a94a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:24.932438 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29941d3f-fec5-4727-a784-5a56351a94a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:16:24.932813 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29941d3f-fec5-4727-a784-5a56351a94a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:24.934487 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29941d3f-fec5-4727-a784-5a56351a94a8 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:24.950693 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1165/2335] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:16:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:16:24.955429 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7bf2ac-f4e0-4372-9ee1-641ad9191bcc None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:24.967318 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7bf2ac-f4e0-4372-9ee1-641ad9191bcc None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:24.974644 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffefb7f4-61d7-4b45-8d3d-206603ab6ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:24.975989 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffefb7f4-61d7-4b45-8d3d-206603ab6ce3 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:24.976642 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffefb7f4-61d7-4b45-8d3d-206603ab6ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:24.977122 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffefb7f4-61d7-4b45-8d3d-206603ab6ce3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:24.978100 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffefb7f4-61d7-4b45-8d3d-206603ab6ce3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:24.980050 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffefb7f4-61d7-4b45-8d3d-206603ab6ce3 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:25.017076 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1171/2336] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:16:24 2022] POST /identity/v3/projects => generated 375 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:16:25.021075 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29941d3f-fec5-4727-a784-5a56351a94a8 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:25.031550 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29941d3f-fec5-4727-a784-5a56351a94a8 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:25.103637 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648e9612-c213-4af7-b37a-f9105b2c3f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:25.104088 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648e9612-c213-4af7-b37a-f9105b2c3f04 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:25.104525 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648e9612-c213-4af7-b37a-f9105b2c3f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:25.104737 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648e9612-c213-4af7-b37a-f9105b2c3f04 admin admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:25.105259 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648e9612-c213-4af7-b37a-f9105b2c3f04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:25.107187 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648e9612-c213-4af7-b37a-f9105b2c3f04 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:25.138968 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1166/2337] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:16:25 2022] POST /identity/v3/users => generated 288 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:16:25.143196 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffefb7f4-61d7-4b45-8d3d-206603ab6ce3 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:25.161551 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffefb7f4-61d7-4b45-8d3d-206603ab6ce3 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:25.167077 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aa8b9a-5924-4b39-bcfc-f6a5a73bfbbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:25.167708 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa8b9a-5924-4b39-bcfc-f6a5a73bfbbf admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:25.168045 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa8b9a-5924-4b39-bcfc-f6a5a73bfbbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:25.168320 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa8b9a-5924-4b39-bcfc-f6a5a73bfbbf admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:25.169012 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aa8b9a-5924-4b39-bcfc-f6a5a73bfbbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:25.170541 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aa8b9a-5924-4b39-bcfc-f6a5a73bfbbf admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:25.182558 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1172/2338] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:16:25 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:16:25.188020 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648e9612-c213-4af7-b37a-f9105b2c3f04 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:25.206317 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648e9612-c213-4af7-b37a-f9105b2c3f04 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:25.211521 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eddeedf-6610-4314-995e-5d7d46b4c80e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:25.212153 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eddeedf-6610-4314-995e-5d7d46b4c80e admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:25.212435 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eddeedf-6610-4314-995e-5d7d46b4c80e admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:25.212710 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eddeedf-6610-4314-995e-5d7d46b4c80e admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:25.213352 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eddeedf-6610-4314-995e-5d7d46b4c80e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:25.214932 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eddeedf-6610-4314-995e-5d7d46b4c80e admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:25.231369 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1167/2339] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:16:25 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:16:25.235649 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aa8b9a-5924-4b39-bcfc-f6a5a73bfbbf admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:25.248528 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aa8b9a-5924-4b39-bcfc-f6a5a73bfbbf admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:25.253994 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a631f8c-8679-4a46-85c3-427d45cae354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:25.254760 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a631f8c-8679-4a46-85c3-427d45cae354 admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:25.255105 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a631f8c-8679-4a46-85c3-427d45cae354 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:25.255481 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a631f8c-8679-4a46-85c3-427d45cae354 admin admin] PATH_INFO: `/v3/projects/9ea0fdf820e748d0814b192a0c5bf572/users/ab6a3226ae7f48838a87c7aafebc35e0/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:25.272137 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a631f8c-8679-4a46-85c3-427d45cae354 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ea0fdf820e748d0814b192a0c5bf572, user_id=ab6a3226ae7f48838a87c7aafebc35e0, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:25.275125 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a631f8c-8679-4a46-85c3-427d45cae354 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:25.277462 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:16:25.297185 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1173/2340] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:16:25 2022] PUT /identity/v3/projects/9ea0fdf820e748d0814b192a0c5bf572/users/ab6a3226ae7f48838a87c7aafebc35e0/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:16:25.310051 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c6ad8b-915f-4106-94af-691625f175a2 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:25.310322 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c6ad8b-915f-4106-94af-691625f175a2 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:25.310597 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c6ad8b-915f-4106-94af-691625f175a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:25.343422 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-c0c6ad8b-915f-4106-94af-691625f175a2 None None] MFA Rules not processed for user `ab6a3226ae7f48838a87c7aafebc35e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:16:25.344859 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-c0c6ad8b-915f-4106-94af-691625f175a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:25.427053 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:16:25.443746 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1168/2341] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:16:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:16:25.455100 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a631f8c-8679-4a46-85c3-427d45cae354 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:25.457896 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-9a631f8c-8679-4a46-85c3-427d45cae354 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:25.563648 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a631f8c-8679-4a46-85c3-427d45cae354 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:25.569439 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b135f16-3abe-4e0a-b26c-613b04a9ee14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:25.570162 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b135f16-3abe-4e0a-b26c-613b04a9ee14 service cinder] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:25.571065 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b135f16-3abe-4e0a-b26c-613b04a9ee14 service cinder] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:25.571369 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b135f16-3abe-4e0a-b26c-613b04a9ee14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:25.581092 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b135f16-3abe-4e0a-b26c-613b04a9ee14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:25.582522 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b135f16-3abe-4e0a-b26c-613b04a9ee14 service cinder] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:25.592051 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1174/2342] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:16:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:16:28.067407 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aab148-c05f-4d5a-8c3d-b7af4d3daba8 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:28.067913 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aab148-c05f-4d5a-8c3d-b7af4d3daba8 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:28.068165 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aab148-c05f-4d5a-8c3d-b7af4d3daba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:28.087497 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-76aab148-c05f-4d5a-8c3d-b7af4d3daba8 None None] MFA Rules not processed for user `ab6a3226ae7f48838a87c7aafebc35e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:16:28.088770 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-76aab148-c05f-4d5a-8c3d-b7af4d3daba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:28.095715 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1169/2343] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:16:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:16:30.652471 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b135f16-3abe-4e0a-b26c-613b04a9ee14 service cinder] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:30.655012 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-9b135f16-3abe-4e0a-b26c-613b04a9ee14 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:30.762703 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b135f16-3abe-4e0a-b26c-613b04a9ee14 service cinder] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:30.771034 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c782d25-deae-4c37-8248-53dc33d60090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:30.771707 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c782d25-deae-4c37-8248-53dc33d60090 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:30.771987 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c782d25-deae-4c37-8248-53dc33d60090 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:30.772256 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c782d25-deae-4c37-8248-53dc33d60090 admin admin] PATH_INFO: `/v3/users/a4d7d1ea5fe3473cb9aa9aff7697c203` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:30.773378 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c782d25-deae-4c37-8248-53dc33d60090 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4d7d1ea5fe3473cb9aa9aff7697c203)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:30.775361 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c782d25-deae-4c37-8248-53dc33d60090 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:30.826198 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-2c782d25-deae-4c37-8248-53dc33d60090 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4d7d1ea5fe3473cb9aa9aff7697c203'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:30.833730 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-2c782d25-deae-4c37-8248-53dc33d60090 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4d7d1ea5fe3473cb9aa9aff7697c203'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:30.834343 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-2c782d25-deae-4c37-8248-53dc33d60090 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4d7d1ea5fe3473cb9aa9aff7697c203'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:30.842440 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-2c782d25-deae-4c37-8248-53dc33d60090 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4d7d1ea5fe3473cb9aa9aff7697c203'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:30.843141 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-2c782d25-deae-4c37-8248-53dc33d60090 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4d7d1ea5fe3473cb9aa9aff7697c203'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:30.860391 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1175/2344] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:16:30 2022] DELETE /identity/v3/users/a4d7d1ea5fe3473cb9aa9aff7697c203 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:16:30.864780 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad51f23-87ff-4847-879f-677c4470842a None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:30.865271 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad51f23-87ff-4847-879f-677c4470842a None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:30.865381 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad51f23-87ff-4847-879f-677c4470842a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:30.885587 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-4ad51f23-87ff-4847-879f-677c4470842a None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:16:30.886913 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-4ad51f23-87ff-4847-879f-677c4470842a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:30.950201 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1170/2345] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:16:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:16:30.961775 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c782d25-deae-4c37-8248-53dc33d60090 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:30.964418 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-2c782d25-deae-4c37-8248-53dc33d60090 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:30.987470 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:16:31.028526 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c782d25-deae-4c37-8248-53dc33d60090 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:31.032198 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9ed2a4-7dbd-46a6-88de-f85334b012c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:31.032793 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9ed2a4-7dbd-46a6-88de-f85334b012c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:31.033070 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9ed2a4-7dbd-46a6-88de-f85334b012c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:31.033339 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9ed2a4-7dbd-46a6-88de-f85334b012c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:31.042613 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9ed2a4-7dbd-46a6-88de-f85334b012c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:31.044068 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9ed2a4-7dbd-46a6-88de-f85334b012c3 service neutron] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:31.053504 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1176/2346] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:16:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:16:31.357405 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad51f23-87ff-4847-879f-677c4470842a None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:31.358994 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-4ad51f23-87ff-4847-879f-677c4470842a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:31.372036 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad51f23-87ff-4847-879f-677c4470842a None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:31.376707 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46880d44-c108-47f2-aa2e-b774d2783857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:31.379102 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46880d44-c108-47f2-aa2e-b774d2783857 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:31.379421 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46880d44-c108-47f2-aa2e-b774d2783857 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:31.379697 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46880d44-c108-47f2-aa2e-b774d2783857 admin admin] PATH_INFO: `/v3/projects/e05176629598478aa5506490c692a50c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:31.380854 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46880d44-c108-47f2-aa2e-b774d2783857 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e05176629598478aa5506490c692a50c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:31.382765 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46880d44-c108-47f2-aa2e-b774d2783857 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:31.440066 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-46880d44-c108-47f2-aa2e-b774d2783857 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e05176629598478aa5506490c692a50c', 'request_id': 'req-46880d44-c108-47f2-aa2e-b774d2783857'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:31.446367 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-46880d44-c108-47f2-aa2e-b774d2783857 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e05176629598478aa5506490c692a50c', 'request_id': 'req-46880d44-c108-47f2-aa2e-b774d2783857'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:31.454116 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-46880d44-c108-47f2-aa2e-b774d2783857 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e05176629598478aa5506490c692a50c', 'request_id': 'req-46880d44-c108-47f2-aa2e-b774d2783857'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:31.465855 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-46880d44-c108-47f2-aa2e-b774d2783857 admin admin] The token cache is being invalidate because project e05176629598478aa5506490c692a50c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:16:31.466076 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-46880d44-c108-47f2-aa2e-b774d2783857 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:31.466725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-46880d44-c108-47f2-aa2e-b774d2783857 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:31.468269 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1171/2347] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:16:31 2022] DELETE /identity/v3/projects/e05176629598478aa5506490c692a50c => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:16:31.473237 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce677f-ee9d-490d-8ef4-d81c08cca34d None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:31.473549 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce677f-ee9d-490d-8ef4-d81c08cca34d None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:31.473830 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce677f-ee9d-490d-8ef4-d81c08cca34d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:31.500454 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-19ce677f-ee9d-490d-8ef4-d81c08cca34d None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:16:31.501670 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-19ce677f-ee9d-490d-8ef4-d81c08cca34d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:31.515731 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46880d44-c108-47f2-aa2e-b774d2783857 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:31.517160 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-46880d44-c108-47f2-aa2e-b774d2783857 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:31.577489 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1177/2348] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:16:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:16:31.585303 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c7d4d-165c-46e4-b365-27dfc6f07fb0 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:31.585597 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c7d4d-165c-46e4-b365-27dfc6f07fb0 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:31.585875 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c7d4d-165c-46e4-b365-27dfc6f07fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:31.599524 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46880d44-c108-47f2-aa2e-b774d2783857 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:31.605043 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e96a813-a54e-425d-b817-adc3012d5685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:31.605814 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e96a813-a54e-425d-b817-adc3012d5685 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:31.606099 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e96a813-a54e-425d-b817-adc3012d5685 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:31.606373 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e96a813-a54e-425d-b817-adc3012d5685 admin admin] PATH_INFO: `/v3/users/710e9c2a16e3484f8d06021d867f1b97` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:31.607551 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e96a813-a54e-425d-b817-adc3012d5685 admin admin] RBAC: Authorizing `identity:delete_user(user_id=710e9c2a16e3484f8d06021d867f1b97)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:31.615364 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-603c7d4d-165c-46e4-b365-27dfc6f07fb0 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:16:31.616894 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e96a813-a54e-425d-b817-adc3012d5685 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:31.618062 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-603c7d4d-165c-46e4-b365-27dfc6f07fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:31.621718 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1178/2349] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:16:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:16:31.625875 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603c7d4d-165c-46e4-b365-27dfc6f07fb0 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:31.630711 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:16:31.636282 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603c7d4d-165c-46e4-b365-27dfc6f07fb0 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:31.641226 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccea3eb2-b3d9-480d-afab-21aafbc64153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:31.641829 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccea3eb2-b3d9-480d-afab-21aafbc64153 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:31.642105 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccea3eb2-b3d9-480d-afab-21aafbc64153 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:31.642376 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccea3eb2-b3d9-480d-afab-21aafbc64153 admin admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:31.643487 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccea3eb2-b3d9-480d-afab-21aafbc64153 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:16:31.644654 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccea3eb2-b3d9-480d-afab-21aafbc64153 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:31.645511 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccea3eb2-b3d9-480d-afab-21aafbc64153 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:31.659360 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1179/2350] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:16:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:16:31.664109 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccea3eb2-b3d9-480d-afab-21aafbc64153 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:31.674957 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccea3eb2-b3d9-480d-afab-21aafbc64153 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:31.682073 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ff50fe-2292-4f2e-b300-e6601fa2b720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:31.682635 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-6e96a813-a54e-425d-b817-adc3012d5685 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '710e9c2a16e3484f8d06021d867f1b97'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:31.683108 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff50fe-2292-4f2e-b300-e6601fa2b720 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:31.683558 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff50fe-2292-4f2e-b300-e6601fa2b720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:31.683991 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff50fe-2292-4f2e-b300-e6601fa2b720 admin admin] PATH_INFO: `/v3/projects` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:31.684874 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ff50fe-2292-4f2e-b300-e6601fa2b720 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:31.687337 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ff50fe-2292-4f2e-b300-e6601fa2b720 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:31.695521 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-6e96a813-a54e-425d-b817-adc3012d5685 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '710e9c2a16e3484f8d06021d867f1b97'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:31.706560 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-6e96a813-a54e-425d-b817-adc3012d5685 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '710e9c2a16e3484f8d06021d867f1b97'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:31.707499 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-6e96a813-a54e-425d-b817-adc3012d5685 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '710e9c2a16e3484f8d06021d867f1b97'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:31.753907 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-6e96a813-a54e-425d-b817-adc3012d5685 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '710e9c2a16e3484f8d06021d867f1b97'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:31.785093 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1172/2351] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:16:31 2022] DELETE /identity/v3/users/710e9c2a16e3484f8d06021d867f1b97 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 12:16:31.789877 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41717391-f3ea-4b28-a2f1-05dcc2c8695b None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:31.790601 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41717391-f3ea-4b28-a2f1-05dcc2c8695b None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:31.791151 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41717391-f3ea-4b28-a2f1-05dcc2c8695b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:31.805170 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1180/2352] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:16:31 2022] POST /identity/v3/projects => generated 383 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 12:16:31.807879 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ff50fe-2292-4f2e-b300-e6601fa2b720 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:31.811042 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-41ff50fe-2292-4f2e-b300-e6601fa2b720 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:31.829301 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-41717391-f3ea-4b28-a2f1-05dcc2c8695b None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:16:31.831808 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-41717391-f3ea-4b28-a2f1-05dcc2c8695b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:31.947714 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ff50fe-2292-4f2e-b300-e6601fa2b720 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:31.958740 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee568a54-d180-4f0b-a737-efaffd914508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:31.959416 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee568a54-d180-4f0b-a737-efaffd914508 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:31.959701 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee568a54-d180-4f0b-a737-efaffd914508 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:31.959976 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee568a54-d180-4f0b-a737-efaffd914508 admin admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:31.960611 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee568a54-d180-4f0b-a737-efaffd914508 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:31.968609 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee568a54-d180-4f0b-a737-efaffd914508 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:31.995570 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1173/2353] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:16:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:16:32.021286 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41717391-f3ea-4b28-a2f1-05dcc2c8695b None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:32.028230 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1181/2354] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:16:31 2022] POST /identity/v3/users => generated 292 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:16:32.034681 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41717391-f3ea-4b28-a2f1-05dcc2c8695b None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:32.041452 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aa78d0-2399-40a0-9577-4277928810d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:32.042500 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aa78d0-2399-40a0-9577-4277928810d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:32.042931 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aa78d0-2399-40a0-9577-4277928810d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:32.043398 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aa78d0-2399-40a0-9577-4277928810d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:32.044217 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3aa78d0-2399-40a0-9577-4277928810d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:32.046531 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3aa78d0-2399-40a0-9577-4277928810d6 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:32.058845 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee568a54-d180-4f0b-a737-efaffd914508 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:32.063741 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-ee568a54-d180-4f0b-a737-efaffd914508 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:32.066449 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1174/2355] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:16:32 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:16:32.070508 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aa78d0-2399-40a0-9577-4277928810d6 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:32.081021 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:16:32.084997 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aa78d0-2399-40a0-9577-4277928810d6 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:32.094418 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39ce96e-18d4-4dad-b2be-455e8e14da9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:32.108722 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ce96e-18d4-4dad-b2be-455e8e14da9c admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:32.109017 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ce96e-18d4-4dad-b2be-455e8e14da9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:32.109297 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ce96e-18d4-4dad-b2be-455e8e14da9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:32.117226 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39ce96e-18d4-4dad-b2be-455e8e14da9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:32.125284 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39ce96e-18d4-4dad-b2be-455e8e14da9c admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:32.140657 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1175/2356] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:16:32 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:16:32.156755 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39ce96e-18d4-4dad-b2be-455e8e14da9c admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:32.157221 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee568a54-d180-4f0b-a737-efaffd914508 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:32.161712 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cde4ae-2be6-4ee3-b6fb-7fb79edf128b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:32.161712 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cde4ae-2be6-4ee3-b6fb-7fb79edf128b service neutron] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:32.161951 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cde4ae-2be6-4ee3-b6fb-7fb79edf128b service neutron] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:32.162201 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cde4ae-2be6-4ee3-b6fb-7fb79edf128b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:32.173112 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39ce96e-18d4-4dad-b2be-455e8e14da9c admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:32.173487 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cde4ae-2be6-4ee3-b6fb-7fb79edf128b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:32.174998 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cde4ae-2be6-4ee3-b6fb-7fb79edf128b service neutron] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:32.180027 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab103a6-8946-4f93-a601-7ce156b50630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:32.181484 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab103a6-8946-4f93-a601-7ce156b50630 admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:32.182084 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab103a6-8946-4f93-a601-7ce156b50630 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:32.182363 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab103a6-8946-4f93-a601-7ce156b50630 admin admin] PATH_INFO: `/v3/projects/1f876d0f217a485392ddcbd79134f609/users/ceed3909469f409189d9ed93937003f0/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:32.189180 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1182/2357] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:16:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 12:16:32.199321 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab103a6-8946-4f93-a601-7ce156b50630 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f876d0f217a485392ddcbd79134f609, user_id=ceed3909469f409189d9ed93937003f0, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:32.202050 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab103a6-8946-4f93-a601-7ce156b50630 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:32.220246 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1176/2358] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:16:32 2022] PUT /identity/v3/projects/1f876d0f217a485392ddcbd79134f609/users/ceed3909469f409189d9ed93937003f0/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:16:32.226903 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4e968-16b4-491f-a71f-48a8dee0f804 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:32.227343 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4e968-16b4-491f-a71f-48a8dee0f804 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:32.227492 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4e968-16b4-491f-a71f-48a8dee0f804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:32.254878 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-69c4e968-16b4-491f-a71f-48a8dee0f804 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:16:32.256559 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-69c4e968-16b4-491f-a71f-48a8dee0f804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:32.332335 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1183/2359] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:16:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:16:32.354609 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab103a6-8946-4f93-a601-7ce156b50630 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:32.357547 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-0ab103a6-8946-4f93-a601-7ce156b50630 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:32.416469 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:16:32.436677 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab103a6-8946-4f93-a601-7ce156b50630 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:32.451880 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b15a46-9a5a-492d-931a-54ee4c954ff4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:32.452616 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b15a46-9a5a-492d-931a-54ee4c954ff4 service neutron] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:32.453634 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b15a46-9a5a-492d-931a-54ee4c954ff4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:32.453916 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b15a46-9a5a-492d-931a-54ee4c954ff4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:32.473325 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b15a46-9a5a-492d-931a-54ee4c954ff4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:32.475831 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b15a46-9a5a-492d-931a-54ee4c954ff4 service neutron] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:32.490083 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1177/2360] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:16:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:16:33.120447 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c4e968-16b4-491f-a71f-48a8dee0f804 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:33.122243 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-69c4e968-16b4-491f-a71f-48a8dee0f804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:33.137977 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c4e968-16b4-491f-a71f-48a8dee0f804 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:33.144404 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0c04a6-1a99-42e5-bb14-cdcf12194b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:33.145523 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0c04a6-1a99-42e5-bb14-cdcf12194b79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:33.145958 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0c04a6-1a99-42e5-bb14-cdcf12194b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:33.146232 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0c04a6-1a99-42e5-bb14-cdcf12194b79 admin admin] PATH_INFO: `/v3/projects/eb95368758194677a6afb4708c6cf6f7` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:33.147820 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0c04a6-1a99-42e5-bb14-cdcf12194b79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb95368758194677a6afb4708c6cf6f7)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:33.150191 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0c04a6-1a99-42e5-bb14-cdcf12194b79 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:33.213641 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-9d0c04a6-1a99-42e5-bb14-cdcf12194b79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb95368758194677a6afb4708c6cf6f7', 'request_id': 'req-9d0c04a6-1a99-42e5-bb14-cdcf12194b79'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:33.222725 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-9d0c04a6-1a99-42e5-bb14-cdcf12194b79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb95368758194677a6afb4708c6cf6f7', 'request_id': 'req-9d0c04a6-1a99-42e5-bb14-cdcf12194b79'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:33.236890 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-9d0c04a6-1a99-42e5-bb14-cdcf12194b79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb95368758194677a6afb4708c6cf6f7', 'request_id': 'req-9d0c04a6-1a99-42e5-bb14-cdcf12194b79'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:33.254356 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-9d0c04a6-1a99-42e5-bb14-cdcf12194b79 admin admin] The token cache is being invalidate because project eb95368758194677a6afb4708c6cf6f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81712) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:16:33.254816 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-9d0c04a6-1a99-42e5-bb14-cdcf12194b79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:33.255655 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-9d0c04a6-1a99-42e5-bb14-cdcf12194b79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:33.260210 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1184/2361] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:16:33 2022] DELETE /identity/v3/projects/eb95368758194677a6afb4708c6cf6f7 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 12:16:33.262880 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2103d3de-925a-41fe-8643-f224bb7b462b None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:33.263361 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2103d3de-925a-41fe-8643-f224bb7b462b None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:33.263675 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2103d3de-925a-41fe-8643-f224bb7b462b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:33.299017 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-2103d3de-925a-41fe-8643-f224bb7b462b None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:16:33.301076 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-2103d3de-925a-41fe-8643-f224bb7b462b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:33.375464 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1178/2362] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:16:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:16:33.384363 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84252488-5f39-4d58-9b8f-bddb6c0c183e None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:33.385001 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84252488-5f39-4d58-9b8f-bddb6c0c183e None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:33.385310 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84252488-5f39-4d58-9b8f-bddb6c0c183e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:33.410305 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-84252488-5f39-4d58-9b8f-bddb6c0c183e None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:16:33.411581 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-84252488-5f39-4d58-9b8f-bddb6c0c183e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:33.427764 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1185/2363] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:16:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:16:33.432692 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2103d3de-925a-41fe-8643-f224bb7b462b None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:33.444509 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2103d3de-925a-41fe-8643-f224bb7b462b None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:33.450124 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329c78fe-eb81-4904-a3bc-03bf4b6605a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:33.450777 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329c78fe-eb81-4904-a3bc-03bf4b6605a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:33.451088 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329c78fe-eb81-4904-a3bc-03bf4b6605a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:33.451539 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329c78fe-eb81-4904-a3bc-03bf4b6605a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:33.452254 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329c78fe-eb81-4904-a3bc-03bf4b6605a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:16:33.452622 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329c78fe-eb81-4904-a3bc-03bf4b6605a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:33.454375 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329c78fe-eb81-4904-a3bc-03bf4b6605a9 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:33.468332 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1179/2364] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:16:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:16:33.473318 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84252488-5f39-4d58-9b8f-bddb6c0c183e None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:33.482630 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84252488-5f39-4d58-9b8f-bddb6c0c183e None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:33.487269 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa996b6-5b77-4b4a-a02e-536d3c8db396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:33.487874 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa996b6-5b77-4b4a-a02e-536d3c8db396 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:33.488151 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa996b6-5b77-4b4a-a02e-536d3c8db396 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:33.488418 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa996b6-5b77-4b4a-a02e-536d3c8db396 admin admin] PATH_INFO: `/v3/projects` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:33.489051 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa996b6-5b77-4b4a-a02e-536d3c8db396 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:33.491207 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa996b6-5b77-4b4a-a02e-536d3c8db396 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:33.526052 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1186/2365] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:16:33 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:16:33.530199 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329c78fe-eb81-4904-a3bc-03bf4b6605a9 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:33.541576 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329c78fe-eb81-4904-a3bc-03bf4b6605a9 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:33.600982 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d91d45-aa2b-4a6b-a782-581738847014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:33.601507 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d91d45-aa2b-4a6b-a782-581738847014 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:33.601812 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d91d45-aa2b-4a6b-a782-581738847014 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:33.602085 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d91d45-aa2b-4a6b-a782-581738847014 admin admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:33.602937 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d91d45-aa2b-4a6b-a782-581738847014 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:33.605520 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d91d45-aa2b-4a6b-a782-581738847014 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:33.631882 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1180/2366] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:16:33 2022] POST /identity/v3/users => generated 286 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 12:16:33.633145 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa996b6-5b77-4b4a-a02e-536d3c8db396 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:33.640150 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:16:33.644343 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa996b6-5b77-4b4a-a02e-536d3c8db396 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:33.651821 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb8db2-47e1-4c98-855e-c1af56073fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:33.652460 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb8db2-47e1-4c98-855e-c1af56073fd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:33.652965 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb8db2-47e1-4c98-855e-c1af56073fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:33.653412 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb8db2-47e1-4c98-855e-c1af56073fd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:33.654305 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdb8db2-47e1-4c98-855e-c1af56073fd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:33.655995 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdb8db2-47e1-4c98-855e-c1af56073fd1 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:33.668159 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1187/2367] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:16:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:16:33.672194 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d91d45-aa2b-4a6b-a782-581738847014 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:33.682787 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d91d45-aa2b-4a6b-a782-581738847014 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:33.687570 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc0fdac-d64f-46fe-bc8e-6482294be6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:33.688187 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc0fdac-d64f-46fe-bc8e-6482294be6de admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:33.688469 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc0fdac-d64f-46fe-bc8e-6482294be6de admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:33.688742 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc0fdac-d64f-46fe-bc8e-6482294be6de admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:33.689391 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc0fdac-d64f-46fe-bc8e-6482294be6de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:33.690991 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc0fdac-d64f-46fe-bc8e-6482294be6de admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:33.702571 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1181/2368] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:16:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:16:33.706982 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb8db2-47e1-4c98-855e-c1af56073fd1 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:33.718181 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb8db2-47e1-4c98-855e-c1af56073fd1 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:33.722904 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b68b4b-855e-4f89-a4a4-abf5dd6bf018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:33.723586 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b68b4b-855e-4f89-a4a4-abf5dd6bf018 admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:33.723864 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b68b4b-855e-4f89-a4a4-abf5dd6bf018 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:33.724160 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b68b4b-855e-4f89-a4a4-abf5dd6bf018 admin admin] PATH_INFO: `/v3/projects/c8c8fa8f81a74460a0a89c5e78505024/users/378105f051844ef3826b47d3d40ff1c5/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:33.748898 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b68b4b-855e-4f89-a4a4-abf5dd6bf018 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8c8fa8f81a74460a0a89c5e78505024, user_id=378105f051844ef3826b47d3d40ff1c5, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:33.754994 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b68b4b-855e-4f89-a4a4-abf5dd6bf018 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:33.776516 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1188/2369] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:16:33 2022] PUT /identity/v3/projects/c8c8fa8f81a74460a0a89c5e78505024/users/378105f051844ef3826b47d3d40ff1c5/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 12:16:33.786774 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfd63d-ac44-4c2f-b58f-6f2e05a0a22c None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:33.787526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfd63d-ac44-4c2f-b58f-6f2e05a0a22c None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:33.787872 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfd63d-ac44-4c2f-b58f-6f2e05a0a22c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:33.828990 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-ebbfd63d-ac44-4c2f-b58f-6f2e05a0a22c None None] MFA Rules not processed for user `378105f051844ef3826b47d3d40ff1c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:16:33.830551 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-ebbfd63d-ac44-4c2f-b58f-6f2e05a0a22c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:33.901138 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:16:33.941389 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1182/2370] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:16:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:16:33.954884 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b68b4b-855e-4f89-a4a4-abf5dd6bf018 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:33.958035 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-f4b68b4b-855e-4f89-a4a4-abf5dd6bf018 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:34.039031 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b68b4b-855e-4f89-a4a4-abf5dd6bf018 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:34.044777 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138914fd-d0d1-4cdc-9471-9195ca60703b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152fd7c708114f249b9c960165f0fdfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:34.045755 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138914fd-d0d1-4cdc-9471-9195ca60703b service glance] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:34.046034 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138914fd-d0d1-4cdc-9471-9195ca60703b service glance] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:34.046306 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138914fd-d0d1-4cdc-9471-9195ca60703b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:34.057795 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138914fd-d0d1-4cdc-9471-9195ca60703b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:34.059689 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138914fd-d0d1-4cdc-9471-9195ca60703b service glance] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:34.071620 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1189/2371] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:16:33 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:16:39.202384 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cd7e3c-41f6-41d8-8f56-2dca49ee597d None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:39.202953 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cd7e3c-41f6-41d8-8f56-2dca49ee597d None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:39.202953 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cd7e3c-41f6-41d8-8f56-2dca49ee597d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:39.258365 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-47cd7e3c-41f6-41d8-8f56-2dca49ee597d None None] MFA Rules not processed for user `ceed3909469f409189d9ed93937003f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:16:39.261507 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-47cd7e3c-41f6-41d8-8f56-2dca49ee597d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:39.373397 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1183/2372] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:16:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:16:39.374565 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138914fd-d0d1-4cdc-9471-9195ca60703b service glance] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:39.376125 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-138914fd-d0d1-4cdc-9471-9195ca60703b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:39.533489 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138914fd-d0d1-4cdc-9471-9195ca60703b service glance] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:39.553327 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9bfbf8-536d-46bb-ba1a-9d0e386b7b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:39.553951 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9bfbf8-536d-46bb-ba1a-9d0e386b7b46 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:39.554231 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9bfbf8-536d-46bb-ba1a-9d0e386b7b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:39.554501 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9bfbf8-536d-46bb-ba1a-9d0e386b7b46 admin admin] PATH_INFO: `/v3/projects` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:39.555147 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9bfbf8-536d-46bb-ba1a-9d0e386b7b46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:39.561002 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9bfbf8-536d-46bb-ba1a-9d0e386b7b46 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:39.617071 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1190/2373] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:16:39 2022] POST /identity/v3/projects => generated 383 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:16:39.621674 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cd7e3c-41f6-41d8-8f56-2dca49ee597d None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:39.636360 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cd7e3c-41f6-41d8-8f56-2dca49ee597d None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:39.737619 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd00b9b-4da1-4362-81e3-113b7321b80e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:39.738271 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd00b9b-4da1-4362-81e3-113b7321b80e admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:39.738511 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd00b9b-4da1-4362-81e3-113b7321b80e admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:39.738783 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd00b9b-4da1-4362-81e3-113b7321b80e admin admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:39.739518 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd00b9b-4da1-4362-81e3-113b7321b80e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:39.741842 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd00b9b-4da1-4362-81e3-113b7321b80e admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:39.769908 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1184/2374] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:16:39 2022] POST /identity/v3/users => generated 298 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 12:16:39.774333 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9bfbf8-536d-46bb-ba1a-9d0e386b7b46 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:39.785550 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9bfbf8-536d-46bb-ba1a-9d0e386b7b46 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:39.790289 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f461091-4055-40ba-9ff8-0e0e1dcfb1a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:39.790909 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f461091-4055-40ba-9ff8-0e0e1dcfb1a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:39.791189 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f461091-4055-40ba-9ff8-0e0e1dcfb1a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:39.791500 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f461091-4055-40ba-9ff8-0e0e1dcfb1a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:39.792187 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f461091-4055-40ba-9ff8-0e0e1dcfb1a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:39.793813 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f461091-4055-40ba-9ff8-0e0e1dcfb1a3 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:39.809902 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1191/2375] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:16:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 12:16:39.814391 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd00b9b-4da1-4362-81e3-113b7321b80e admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:39.827030 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd00b9b-4da1-4362-81e3-113b7321b80e admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:39.832931 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2e6100-b59f-4f74-b622-6dab8ac645f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:39.833870 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2e6100-b59f-4f74-b622-6dab8ac645f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:39.834155 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2e6100-b59f-4f74-b622-6dab8ac645f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:39.835750 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2e6100-b59f-4f74-b622-6dab8ac645f6 admin admin] PATH_INFO: `/v3/projects/44cb1c468c464288bd570487f79770fd/users/2e38dc9b4d0c4308a355442328911615/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:39.851652 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2e6100-b59f-4f74-b622-6dab8ac645f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44cb1c468c464288bd570487f79770fd, user_id=2e38dc9b4d0c4308a355442328911615, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:39.854535 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2e6100-b59f-4f74-b622-6dab8ac645f6 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:39.872063 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1185/2376] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:16:39 2022] PUT /identity/v3/projects/44cb1c468c464288bd570487f79770fd/users/2e38dc9b4d0c4308a355442328911615/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:16:39.876136 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f461091-4055-40ba-9ff8-0e0e1dcfb1a3 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:39.889405 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f461091-4055-40ba-9ff8-0e0e1dcfb1a3 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:39.942473 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:16:39.957989 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6f66a6-a991-407a-83d4-61d321fb6d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:39.962276 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f66a6-a991-407a-83d4-61d321fb6d41 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:39.962276 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f66a6-a991-407a-83d4-61d321fb6d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:39.962276 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f66a6-a991-407a-83d4-61d321fb6d41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:39.962276 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6f66a6-a991-407a-83d4-61d321fb6d41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:39.962276 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6f66a6-a991-407a-83d4-61d321fb6d41 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:39.976867 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1192/2377] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:16:39 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:16:39.978776 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2e6100-b59f-4f74-b622-6dab8ac645f6 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:39.990988 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2e6100-b59f-4f74-b622-6dab8ac645f6 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:39.996898 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909ff936-a067-4fbd-8abd-543835a227e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:39.997714 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ff936-a067-4fbd-8abd-543835a227e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:39.998104 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ff936-a067-4fbd-8abd-543835a227e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:39.998482 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ff936-a067-4fbd-8abd-543835a227e4 admin admin] PATH_INFO: `/v3/projects/44cb1c468c464288bd570487f79770fd/users/2e38dc9b4d0c4308a355442328911615/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:40.000591 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909ff936-a067-4fbd-8abd-543835a227e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44cb1c468c464288bd570487f79770fd, user_id=2e38dc9b4d0c4308a355442328911615, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:40.003308 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909ff936-a067-4fbd-8abd-543835a227e4 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:40.018159 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1186/2378] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:16:39 2022] PUT /identity/v3/projects/44cb1c468c464288bd570487f79770fd/users/2e38dc9b4d0c4308a355442328911615/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:16:40.280571 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27af495-cf7f-4e22-8a4f-1929de5e973c None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:40.280859 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27af495-cf7f-4e22-8a4f-1929de5e973c None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:40.281141 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27af495-cf7f-4e22-8a4f-1929de5e973c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:40.349412 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-b27af495-cf7f-4e22-8a4f-1929de5e973c None None] MFA Rules not processed for user `a02508c20bd547708638c1875101da6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:16:40.350907 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-b27af495-cf7f-4e22-8a4f-1929de5e973c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:40.459071 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1193/2379] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:16:40 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:16:47.108609 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef3d48-a46d-48b7-92b7-9e3ef5fbf0d0 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:47.109073 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef3d48-a46d-48b7-92b7-9e3ef5fbf0d0 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:47.109241 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef3d48-a46d-48b7-92b7-9e3ef5fbf0d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:47.151172 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-d1ef3d48-a46d-48b7-92b7-9e3ef5fbf0d0 None None] MFA Rules not processed for user `2e38dc9b4d0c4308a355442328911615`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:16:47.152440 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-d1ef3d48-a46d-48b7-92b7-9e3ef5fbf0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:47.191555 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:16:47.307821 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1187/2380] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:16:47 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:16:47.325257 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27af495-cf7f-4e22-8a4f-1929de5e973c None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:47.414721 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27af495-cf7f-4e22-8a4f-1929de5e973c None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:47.420407 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde7c88e-52b2-40f8-9c49-d546dded3f41 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152fd7c708114f249b9c960165f0fdfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:47.420692 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde7c88e-52b2-40f8-9c49-d546dded3f41 service glance] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:47.421257 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde7c88e-52b2-40f8-9c49-d546dded3f41 service glance] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:47.421603 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde7c88e-52b2-40f8-9c49-d546dded3f41 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:47.523244 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde7c88e-52b2-40f8-9c49-d546dded3f41 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:47.525956 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde7c88e-52b2-40f8-9c49-d546dded3f41 service glance] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:47.536907 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1194/2381] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:16:47 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:16:47.854601 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f5ab5-f61d-4c0c-81a2-8a422697510b None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:47.855117 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f5ab5-f61d-4c0c-81a2-8a422697510b None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:47.855831 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f5ab5-f61d-4c0c-81a2-8a422697510b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:47.936719 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-f50f5ab5-f61d-4c0c-81a2-8a422697510b None None] MFA Rules not processed for user `755b04cae45746569bfbf92f2f213a58`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:16:47.939423 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-f50f5ab5-f61d-4c0c-81a2-8a422697510b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:48.059317 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1188/2382] 10.222.0.61 () {58 vars in 1009 bytes} [Fri Aug 26 12:16:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:16:48.140323 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde7c88e-52b2-40f8-9c49-d546dded3f41 service glance] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:48.142818 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-dde7c88e-52b2-40f8-9c49-d546dded3f41 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:48.262658 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:16:48.336419 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde7c88e-52b2-40f8-9c49-d546dded3f41 service glance] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:48.346687 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89fe1be-63ef-4b33-b56c-2800dcec323f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead95a244d11432d9fe9f718e1a75f38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:48.347717 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89fe1be-63ef-4b33-b56c-2800dcec323f service swift] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:48.348169 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89fe1be-63ef-4b33-b56c-2800dcec323f service swift] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:48.348596 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89fe1be-63ef-4b33-b56c-2800dcec323f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:48.361060 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89fe1be-63ef-4b33-b56c-2800dcec323f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:48.363336 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89fe1be-63ef-4b33-b56c-2800dcec323f service swift] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:48.415679 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1195/2383] 10.222.0.61 () {60 vars in 1384 bytes} [Fri Aug 26 12:16:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 12:16:51.146266 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50f5ab5-f61d-4c0c-81a2-8a422697510b None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:51.148308 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-f50f5ab5-f61d-4c0c-81a2-8a422697510b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:51.164168 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e921f5d3-59e9-4316-8565-8fa2510be81e None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:51.164759 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e921f5d3-59e9-4316-8565-8fa2510be81e None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:51.164759 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e921f5d3-59e9-4316-8565-8fa2510be81e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:51.200540 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-e921f5d3-59e9-4316-8565-8fa2510be81e None None] MFA Rules not processed for user `755b04cae45746569bfbf92f2f213a58`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:16:51.205348 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-e921f5d3-59e9-4316-8565-8fa2510be81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:51.228191 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1196/2384] 10.222.0.61 () {58 vars in 1009 bytes} [Fri Aug 26 12:16:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:16:51.254727 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e921f5d3-59e9-4316-8565-8fa2510be81e None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:51.272850 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50f5ab5-f61d-4c0c-81a2-8a422697510b None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:51.276282 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e921f5d3-59e9-4316-8565-8fa2510be81e None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:51.277693 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f45062-0524-442d-92b7-2bd16fd8afc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:51.278319 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f45062-0524-442d-92b7-2bd16fd8afc3 service neutron] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:51.278608 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f45062-0524-442d-92b7-2bd16fd8afc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:51.278986 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f45062-0524-442d-92b7-2bd16fd8afc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:51.280923 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-b8f45062-0524-442d-92b7-2bd16fd8afc3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:51.303794 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2997ddde-56a8-42c7-b96d-8ac42efbaae0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead95a244d11432d9fe9f718e1a75f38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:51.305242 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2997ddde-56a8-42c7-b96d-8ac42efbaae0 service swift] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:51.306345 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2997ddde-56a8-42c7-b96d-8ac42efbaae0 service swift] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:51.307027 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2997ddde-56a8-42c7-b96d-8ac42efbaae0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:51.323363 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2997ddde-56a8-42c7-b96d-8ac42efbaae0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:51.324962 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2997ddde-56a8-42c7-b96d-8ac42efbaae0 service swift] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:51.347420 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1197/2385] 10.222.0.61 () {60 vars in 1384 bytes} [Fri Aug 26 12:16:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 26 12:16:51.429732 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f45062-0524-442d-92b7-2bd16fd8afc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:51.431999 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f45062-0524-442d-92b7-2bd16fd8afc3 service neutron] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:51.452379 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ecf2c-e1a0-48f7-8b80-259b1229458f None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:51.453287 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ecf2c-e1a0-48f7-8b80-259b1229458f None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:51.453882 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ecf2c-e1a0-48f7-8b80-259b1229458f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:51.461187 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1189/2386] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:16:51 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:16:51.517262 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-e28ecf2c-e1a0-48f7-8b80-259b1229458f None None] MFA Rules not processed for user `36a27ca98a2e4521b269be48c9ec4cb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:16:51.522566 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-e28ecf2c-e1a0-48f7-8b80-259b1229458f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:51.682187 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1198/2387] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:16:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:16:52.678449 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489fcaa3-68f0-4048-9f58-70df431afcc6 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:52.679013 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489fcaa3-68f0-4048-9f58-70df431afcc6 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:52.679013 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489fcaa3-68f0-4048-9f58-70df431afcc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:52.691316 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:16:52.711481 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-489fcaa3-68f0-4048-9f58-70df431afcc6 None None] MFA Rules not processed for user `755b04cae45746569bfbf92f2f213a58`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:16:52.715074 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-489fcaa3-68f0-4048-9f58-70df431afcc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:52.738115 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1190/2388] 10.222.0.61 () {58 vars in 967 bytes} [Fri Aug 26 12:16:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:16:52.762213 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28ecf2c-e1a0-48f7-8b80-259b1229458f None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:52.777355 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28ecf2c-e1a0-48f7-8b80-259b1229458f None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:52.788498 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d2209f-85f3-4bec-b5b8-6a8479cf353a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead95a244d11432d9fe9f718e1a75f38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:52.789665 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d2209f-85f3-4bec-b5b8-6a8479cf353a service swift] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:52.790711 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d2209f-85f3-4bec-b5b8-6a8479cf353a service swift] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:52.791376 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d2209f-85f3-4bec-b5b8-6a8479cf353a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:52.802759 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d2209f-85f3-4bec-b5b8-6a8479cf353a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:52.805560 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d2209f-85f3-4bec-b5b8-6a8479cf353a service swift] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:52.816239 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1199/2389] 10.222.0.61 () {60 vars in 1384 bytes} [Fri Aug 26 12:16:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 12:16:53.410551 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06106a29-22c4-4ad6-8a4c-69dade23d542 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:53.410911 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06106a29-22c4-4ad6-8a4c-69dade23d542 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:53.411306 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06106a29-22c4-4ad6-8a4c-69dade23d542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:53.447031 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-06106a29-22c4-4ad6-8a4c-69dade23d542 None None] MFA Rules not processed for user `755b04cae45746569bfbf92f2f213a58`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:16:53.450542 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-06106a29-22c4-4ad6-8a4c-69dade23d542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:53.481579 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1191/2390] 10.222.0.61 () {58 vars in 1009 bytes} [Fri Aug 26 12:16:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:16:53.548137 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d2209f-85f3-4bec-b5b8-6a8479cf353a service swift] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:53.567437 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d2209f-85f3-4bec-b5b8-6a8479cf353a service swift] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:53.597553 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be04cea0-ea73-46ec-9d23-411c0f1710a6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead95a244d11432d9fe9f718e1a75f38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:53.598333 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be04cea0-ea73-46ec-9d23-411c0f1710a6 service swift] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:53.599442 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be04cea0-ea73-46ec-9d23-411c0f1710a6 service swift] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:53.599808 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be04cea0-ea73-46ec-9d23-411c0f1710a6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:53.634929 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be04cea0-ea73-46ec-9d23-411c0f1710a6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:53.636640 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be04cea0-ea73-46ec-9d23-411c0f1710a6 service swift] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:53.681977 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1200/2391] 10.222.0.61 () {60 vars in 1384 bytes} [Fri Aug 26 12:16:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 12:16:54.694472 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5cd3f9-4122-45a1-949d-2676fbb3a2f1 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:54.695017 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5cd3f9-4122-45a1-949d-2676fbb3a2f1 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:54.695017 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5cd3f9-4122-45a1-949d-2676fbb3a2f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:54.722429 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-1d5cd3f9-4122-45a1-949d-2676fbb3a2f1 None None] MFA Rules not processed for user `755b04cae45746569bfbf92f2f213a58`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:16:54.724627 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-1d5cd3f9-4122-45a1-949d-2676fbb3a2f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:54.737776 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1192/2392] 10.222.0.61 () {58 vars in 967 bytes} [Fri Aug 26 12:16:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:16:54.766775 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be04cea0-ea73-46ec-9d23-411c0f1710a6 service swift] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:54.804467 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be04cea0-ea73-46ec-9d23-411c0f1710a6 service swift] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:54.827650 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3cf4f9-63e5-4a62-b743-7b5563c8b6cd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead95a244d11432d9fe9f718e1a75f38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:54.828265 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3cf4f9-63e5-4a62-b743-7b5563c8b6cd service swift] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:54.828713 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3cf4f9-63e5-4a62-b743-7b5563c8b6cd service swift] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:54.829128 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3cf4f9-63e5-4a62-b743-7b5563c8b6cd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:54.846752 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3cf4f9-63e5-4a62-b743-7b5563c8b6cd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:54.860249 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3cf4f9-63e5-4a62-b743-7b5563c8b6cd service swift] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:54.906254 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1201/2393] 10.222.0.61 () {60 vars in 1384 bytes} [Fri Aug 26 12:16:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 12:16:59.101338 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5cd3f9-4122-45a1-949d-2676fbb3a2f1 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:59.103761 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-1d5cd3f9-4122-45a1-949d-2676fbb3a2f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:59.176898 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5cd3f9-4122-45a1-949d-2676fbb3a2f1 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:59.184877 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079bb885-3bbd-446d-98e3-1ee225f86ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:59.185534 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079bb885-3bbd-446d-98e3-1ee225f86ab4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:59.186040 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079bb885-3bbd-446d-98e3-1ee225f86ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:59.186324 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079bb885-3bbd-446d-98e3-1ee225f86ab4 admin admin] PATH_INFO: `/v3/users/ab6a3226ae7f48838a87c7aafebc35e0` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:59.187544 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079bb885-3bbd-446d-98e3-1ee225f86ab4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab6a3226ae7f48838a87c7aafebc35e0)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:59.189508 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079bb885-3bbd-446d-98e3-1ee225f86ab4 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:59.248799 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-079bb885-3bbd-446d-98e3-1ee225f86ab4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab6a3226ae7f48838a87c7aafebc35e0'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:59.258773 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-079bb885-3bbd-446d-98e3-1ee225f86ab4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab6a3226ae7f48838a87c7aafebc35e0'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:59.269514 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-079bb885-3bbd-446d-98e3-1ee225f86ab4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab6a3226ae7f48838a87c7aafebc35e0'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:59.270274 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-079bb885-3bbd-446d-98e3-1ee225f86ab4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab6a3226ae7f48838a87c7aafebc35e0'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:59.289455 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-079bb885-3bbd-446d-98e3-1ee225f86ab4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab6a3226ae7f48838a87c7aafebc35e0'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:59.302882 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1193/2394] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:16:59 2022] DELETE /identity/v3/users/ab6a3226ae7f48838a87c7aafebc35e0 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:16:59.304172 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0cc75-406e-48f4-942a-d93498192436 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:59.304664 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0cc75-406e-48f4-942a-d93498192436 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:59.304747 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0cc75-406e-48f4-942a-d93498192436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:59.340089 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-77f0cc75-406e-48f4-942a-d93498192436 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:16:59.341589 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-77f0cc75-406e-48f4-942a-d93498192436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:59.446924 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1202/2395] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:16:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:16:59.454319 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079bb885-3bbd-446d-98e3-1ee225f86ab4 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:59.455909 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-079bb885-3bbd-446d-98e3-1ee225f86ab4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:59.573158 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079bb885-3bbd-446d-98e3-1ee225f86ab4 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:59.577573 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee12368-c287-45a1-b3a5-637c1b29ca3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:59.578363 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee12368-c287-45a1-b3a5-637c1b29ca3d service neutron] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:59.578757 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee12368-c287-45a1-b3a5-637c1b29ca3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:59.579137 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee12368-c287-45a1-b3a5-637c1b29ca3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:59.592450 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee12368-c287-45a1-b3a5-637c1b29ca3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:59.595398 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee12368-c287-45a1-b3a5-637c1b29ca3d service neutron] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:59.610379 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1194/2396] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:16:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 12:16:59.839768 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f0cc75-406e-48f4-942a-d93498192436 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:16:59.841247 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-77f0cc75-406e-48f4-942a-d93498192436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:16:59.859941 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f0cc75-406e-48f4-942a-d93498192436 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:16:59.870305 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28df7c1e-5aa7-4884-859e-7b1f8b36b2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:16:59.870305 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df7c1e-5aa7-4884-859e-7b1f8b36b2de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:59.870305 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df7c1e-5aa7-4884-859e-7b1f8b36b2de admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:59.870305 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df7c1e-5aa7-4884-859e-7b1f8b36b2de admin admin] PATH_INFO: `/v3/projects/9ea0fdf820e748d0814b192a0c5bf572` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:16:59.873074 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28df7c1e-5aa7-4884-859e-7b1f8b36b2de admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ea0fdf820e748d0814b192a0c5bf572)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:16:59.874672 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28df7c1e-5aa7-4884-859e-7b1f8b36b2de admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:16:59.891868 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:16:59.951028 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-28df7c1e-5aa7-4884-859e-7b1f8b36b2de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ea0fdf820e748d0814b192a0c5bf572', 'request_id': 'req-28df7c1e-5aa7-4884-859e-7b1f8b36b2de'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:59.958103 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-28df7c1e-5aa7-4884-859e-7b1f8b36b2de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ea0fdf820e748d0814b192a0c5bf572', 'request_id': 'req-28df7c1e-5aa7-4884-859e-7b1f8b36b2de'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:59.970761 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-28df7c1e-5aa7-4884-859e-7b1f8b36b2de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ea0fdf820e748d0814b192a0c5bf572', 'request_id': 'req-28df7c1e-5aa7-4884-859e-7b1f8b36b2de'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:59.984164 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-28df7c1e-5aa7-4884-859e-7b1f8b36b2de admin admin] The token cache is being invalidate because project 9ea0fdf820e748d0814b192a0c5bf572 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81712) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:16:59.984560 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-28df7c1e-5aa7-4884-859e-7b1f8b36b2de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:59.985279 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-28df7c1e-5aa7-4884-859e-7b1f8b36b2de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:16:59.986845 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1203/2397] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:16:59 2022] DELETE /identity/v3/projects/9ea0fdf820e748d0814b192a0c5bf572 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:16:59.992211 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df871abb-e760-4ad6-862d-d9df72ab9666 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:16:59.992628 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df871abb-e760-4ad6-862d-d9df72ab9666 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:16:59.993004 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df871abb-e760-4ad6-862d-d9df72ab9666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:17:00.015555 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:17:00.047600 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-df871abb-e760-4ad6-862d-d9df72ab9666 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:17:00.049855 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-df871abb-e760-4ad6-862d-d9df72ab9666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:17:00.179663 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1195/2398] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:16:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:17:00.189546 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cfdc24-7165-45ae-9a68-5e58040f7f35 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:17:00.190104 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cfdc24-7165-45ae-9a68-5e58040f7f35 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:17:00.190584 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cfdc24-7165-45ae-9a68-5e58040f7f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:17:00.232948 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-43cfdc24-7165-45ae-9a68-5e58040f7f35 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:17:00.234704 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-43cfdc24-7165-45ae-9a68-5e58040f7f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:17:00.254974 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1204/2399] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:17:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:17:00.259325 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df871abb-e760-4ad6-862d-d9df72ab9666 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:17:00.274007 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df871abb-e760-4ad6-862d-d9df72ab9666 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:17:00.283263 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6a173d-37ef-459a-987d-12af3f782ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:17:00.284088 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a173d-37ef-459a-987d-12af3f782ed8 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:17:00.285022 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a173d-37ef-459a-987d-12af3f782ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:17:00.285439 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6a173d-37ef-459a-987d-12af3f782ed8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:17:00.286573 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6a173d-37ef-459a-987d-12af3f782ed8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:17:00.287195 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6a173d-37ef-459a-987d-12af3f782ed8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:17:00.291264 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6a173d-37ef-459a-987d-12af3f782ed8 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:17:00.307084 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1196/2400] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:17:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:17:00.312235 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cfdc24-7165-45ae-9a68-5e58040f7f35 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:17:00.322475 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cfdc24-7165-45ae-9a68-5e58040f7f35 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:17:00.330008 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c16ef86-6e42-48d2-a9c4-5046c98f5992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:17:00.330997 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c16ef86-6e42-48d2-a9c4-5046c98f5992 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:17:00.331383 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c16ef86-6e42-48d2-a9c4-5046c98f5992 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:17:00.331750 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c16ef86-6e42-48d2-a9c4-5046c98f5992 admin admin] PATH_INFO: `/v3/projects` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:17:00.334098 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c16ef86-6e42-48d2-a9c4-5046c98f5992 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:17:00.336338 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c16ef86-6e42-48d2-a9c4-5046c98f5992 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:17:00.382149 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6a173d-37ef-459a-987d-12af3f782ed8 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:17:00.393766 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1205/2401] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:17:00 2022] POST /identity/v3/projects => generated 383 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:17:00.396646 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6a173d-37ef-459a-987d-12af3f782ed8 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:17:00.486944 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2d0fe1-6ac0-4788-b45e-dc1a1ede9bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:17:00.489486 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d0fe1-6ac0-4788-b45e-dc1a1ede9bed admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:17:00.489486 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d0fe1-6ac0-4788-b45e-dc1a1ede9bed admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:17:00.490406 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d0fe1-6ac0-4788-b45e-dc1a1ede9bed admin admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:17:00.491752 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2d0fe1-6ac0-4788-b45e-dc1a1ede9bed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:17:00.495545 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2d0fe1-6ac0-4788-b45e-dc1a1ede9bed admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:17:00.534285 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c16ef86-6e42-48d2-a9c4-5046c98f5992 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:17:00.545740 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1197/2402] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:17:00 2022] POST /identity/v3/users => generated 292 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:17:00.548813 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c16ef86-6e42-48d2-a9c4-5046c98f5992 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:17:00.557762 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19bc715-a179-4e33-9031-4c8cadc6bfdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:17:00.558383 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19bc715-a179-4e33-9031-4c8cadc6bfdb admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:17:00.558665 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19bc715-a179-4e33-9031-4c8cadc6bfdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:17:00.559099 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19bc715-a179-4e33-9031-4c8cadc6bfdb admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:17:00.559906 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19bc715-a179-4e33-9031-4c8cadc6bfdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:17:00.561786 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19bc715-a179-4e33-9031-4c8cadc6bfdb admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:17:00.584955 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2d0fe1-6ac0-4788-b45e-dc1a1ede9bed admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:17:00.585349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1206/2403] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:17:00 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:17:00.601386 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2d0fe1-6ac0-4788-b45e-dc1a1ede9bed admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:17:00.611417 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc236f-5c68-43ee-89d4-6452c3c55619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:17:00.612046 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc236f-5c68-43ee-89d4-6452c3c55619 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:17:00.612327 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc236f-5c68-43ee-89d4-6452c3c55619 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:17:00.612598 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc236f-5c68-43ee-89d4-6452c3c55619 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:17:00.613241 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bc236f-5c68-43ee-89d4-6452c3c55619 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:17:00.615115 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bc236f-5c68-43ee-89d4-6452c3c55619 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:17:00.636430 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1198/2404] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:17:00 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:17:00.637331 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19bc715-a179-4e33-9031-4c8cadc6bfdb admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:17:00.656406 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19bc715-a179-4e33-9031-4c8cadc6bfdb admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:17:00.661420 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2a55ce-a183-48de-8360-541fd89ae9d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:17:00.662158 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2a55ce-a183-48de-8360-541fd89ae9d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:17:00.662575 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2a55ce-a183-48de-8360-541fd89ae9d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:17:00.662908 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2a55ce-a183-48de-8360-541fd89ae9d9 admin admin] PATH_INFO: `/v3/projects/2ab22ea009914891b1f594ed3d1e4ec3/users/a7d0a1ee39274a0d90a2be9fbc26a58b/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:17:00.688980 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2a55ce-a183-48de-8360-541fd89ae9d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ab22ea009914891b1f594ed3d1e4ec3, user_id=a7d0a1ee39274a0d90a2be9fbc26a58b, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:17:00.692989 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2a55ce-a183-48de-8360-541fd89ae9d9 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:17:00.732534 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1207/2405] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:17:00 2022] PUT /identity/v3/projects/2ab22ea009914891b1f594ed3d1e4ec3/users/a7d0a1ee39274a0d90a2be9fbc26a58b/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 12:17:00.739254 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2636c-1ccd-4b50-97ab-54a27789cb7e None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:17:00.739895 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2636c-1ccd-4b50-97ab-54a27789cb7e None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:17:00.740792 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2636c-1ccd-4b50-97ab-54a27789cb7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:17:00.774884 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-68c2636c-1ccd-4b50-97ab-54a27789cb7e None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:17:00.776410 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-68c2636c-1ccd-4b50-97ab-54a27789cb7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:17:00.883195 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1199/2406] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:17:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:17:00.897807 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2a55ce-a183-48de-8360-541fd89ae9d9 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:17:00.902261 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-3f2a55ce-a183-48de-8360-541fd89ae9d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:17:01.062477 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2a55ce-a183-48de-8360-541fd89ae9d9 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:17:01.073755 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44719508-c377-4bcb-8ec3-c3e96b302b22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:17:01.076291 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44719508-c377-4bcb-8ec3-c3e96b302b22 service neutron] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:17:01.076291 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44719508-c377-4bcb-8ec3-c3e96b302b22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:17:01.076291 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44719508-c377-4bcb-8ec3-c3e96b302b22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:17:01.092859 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44719508-c377-4bcb-8ec3-c3e96b302b22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:17:01.096471 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44719508-c377-4bcb-8ec3-c3e96b302b22 service neutron] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:17:01.113695 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1208/2407] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:17:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:17:03.908522 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e433ef45-c9c6-4fc7-98bc-45866b767ac1 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:17:03.909175 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e433ef45-c9c6-4fc7-98bc-45866b767ac1 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:17:03.920252 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e433ef45-c9c6-4fc7-98bc-45866b767ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:17:03.969629 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:17:03.971791 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3088afa2-58bb-4f93-9a05-e3799ca2b9ec None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:17:03.972303 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3088afa2-58bb-4f93-9a05-e3799ca2b9ec None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:17:03.972419 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3088afa2-58bb-4f93-9a05-e3799ca2b9ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:17:03.975656 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-e433ef45-c9c6-4fc7-98bc-45866b767ac1 None None] MFA Rules not processed for user `a7d0a1ee39274a0d90a2be9fbc26a58b`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:17:03.978710 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-e433ef45-c9c6-4fc7-98bc-45866b767ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:17:04.000998 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-3088afa2-58bb-4f93-9a05-e3799ca2b9ec None None] MFA Rules not processed for user `ceed3909469f409189d9ed93937003f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:17:04.002309 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-3088afa2-58bb-4f93-9a05-e3799ca2b9ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:17:04.048200 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:17:04.078325 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1209/2408] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:17:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:17:04.086821 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1200/2409] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:17:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:17:04.094445 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3088afa2-58bb-4f93-9a05-e3799ca2b9ec None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:17:04.095881 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-3088afa2-58bb-4f93-9a05-e3799ca2b9ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:17:04.159355 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3088afa2-58bb-4f93-9a05-e3799ca2b9ec None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:17:04.163004 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a80ef03-e78b-48ad-8293-b920baf6d90a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:17:04.163643 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a80ef03-e78b-48ad-8293-b920baf6d90a service cinder] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:17:04.163919 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a80ef03-e78b-48ad-8293-b920baf6d90a service cinder] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:17:04.164187 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a80ef03-e78b-48ad-8293-b920baf6d90a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:17:04.174674 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a80ef03-e78b-48ad-8293-b920baf6d90a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:17:04.176141 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a80ef03-e78b-48ad-8293-b920baf6d90a service cinder] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:17:04.186054 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1210/2410] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:17:04 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:17:04.777890 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e433ef45-c9c6-4fc7-98bc-45866b767ac1 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:17:04.779446 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-e433ef45-c9c6-4fc7-98bc-45866b767ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:17:04.794816 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e433ef45-c9c6-4fc7-98bc-45866b767ac1 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:17:04.809102 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd171c5-f325-4405-9884-a1e17f2875cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:17:04.809811 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd171c5-f325-4405-9884-a1e17f2875cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:17:04.810187 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd171c5-f325-4405-9884-a1e17f2875cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:17:04.810690 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd171c5-f325-4405-9884-a1e17f2875cf admin admin] PATH_INFO: `/v3/users/378105f051844ef3826b47d3d40ff1c5` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:17:04.812013 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd171c5-f325-4405-9884-a1e17f2875cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=378105f051844ef3826b47d3d40ff1c5)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:17:04.814453 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd171c5-f325-4405-9884-a1e17f2875cf admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:17:04.908174 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-3cd171c5-f325-4405-9884-a1e17f2875cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '378105f051844ef3826b47d3d40ff1c5'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:17:04.935482 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-3cd171c5-f325-4405-9884-a1e17f2875cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '378105f051844ef3826b47d3d40ff1c5'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:17:04.947113 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-3cd171c5-f325-4405-9884-a1e17f2875cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '378105f051844ef3826b47d3d40ff1c5'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:17:04.947946 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-3cd171c5-f325-4405-9884-a1e17f2875cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '378105f051844ef3826b47d3d40ff1c5'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:17:04.965100 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-3cd171c5-f325-4405-9884-a1e17f2875cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '378105f051844ef3826b47d3d40ff1c5'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:17:04.972857 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1201/2411] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:17:04 2022] DELETE /identity/v3/users/378105f051844ef3826b47d3d40ff1c5 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:17:04.978088 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcfbc7f-2ffc-4a7d-8b2b-f26fea2dbf69 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:17:04.978673 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcfbc7f-2ffc-4a7d-8b2b-f26fea2dbf69 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:17:04.978757 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcfbc7f-2ffc-4a7d-8b2b-f26fea2dbf69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:17:05.002013 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-bdcfbc7f-2ffc-4a7d-8b2b-f26fea2dbf69 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:17:05.003230 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-bdcfbc7f-2ffc-4a7d-8b2b-f26fea2dbf69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:17:05.119102 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1211/2412] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:17:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:17:05.120695 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd171c5-f325-4405-9884-a1e17f2875cf admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:17:05.122776 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-3cd171c5-f325-4405-9884-a1e17f2875cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:17:05.202578 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd171c5-f325-4405-9884-a1e17f2875cf admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:17:05.206550 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfe72e5-030c-4f03-aec8-bf23c76360af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:17:05.207304 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfe72e5-030c-4f03-aec8-bf23c76360af service neutron] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:17:05.207583 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfe72e5-030c-4f03-aec8-bf23c76360af service neutron] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:17:05.207861 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfe72e5-030c-4f03-aec8-bf23c76360af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:17:05.219481 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfe72e5-030c-4f03-aec8-bf23c76360af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:17:05.221070 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfe72e5-030c-4f03-aec8-bf23c76360af service neutron] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:17:05.235034 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1202/2413] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:17:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 12:17:05.543144 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcfbc7f-2ffc-4a7d-8b2b-f26fea2dbf69 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:17:05.545019 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-bdcfbc7f-2ffc-4a7d-8b2b-f26fea2dbf69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:17:05.557644 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcfbc7f-2ffc-4a7d-8b2b-f26fea2dbf69 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:17:05.562283 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7523e573-c812-4cab-90fb-41fc312a4d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:17:05.562992 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7523e573-c812-4cab-90fb-41fc312a4d9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:17:05.563370 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7523e573-c812-4cab-90fb-41fc312a4d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:17:05.564275 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7523e573-c812-4cab-90fb-41fc312a4d9f admin admin] PATH_INFO: `/v3/projects/c8c8fa8f81a74460a0a89c5e78505024` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:17:05.565403 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7523e573-c812-4cab-90fb-41fc312a4d9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8c8fa8f81a74460a0a89c5e78505024)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:17:05.567500 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7523e573-c812-4cab-90fb-41fc312a4d9f admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:17:05.623981 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-7523e573-c812-4cab-90fb-41fc312a4d9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8c8fa8f81a74460a0a89c5e78505024', 'request_id': 'req-7523e573-c812-4cab-90fb-41fc312a4d9f'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:17:05.631033 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-7523e573-c812-4cab-90fb-41fc312a4d9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8c8fa8f81a74460a0a89c5e78505024', 'request_id': 'req-7523e573-c812-4cab-90fb-41fc312a4d9f'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:17:05.641126 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-7523e573-c812-4cab-90fb-41fc312a4d9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8c8fa8f81a74460a0a89c5e78505024', 'request_id': 'req-7523e573-c812-4cab-90fb-41fc312a4d9f'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:17:05.655800 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-7523e573-c812-4cab-90fb-41fc312a4d9f admin admin] The token cache is being invalidate because project c8c8fa8f81a74460a0a89c5e78505024 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81712) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:17:05.656487 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-7523e573-c812-4cab-90fb-41fc312a4d9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:17:05.657240 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-7523e573-c812-4cab-90fb-41fc312a4d9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:17:05.659286 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1212/2414] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:17:05 2022] DELETE /identity/v3/projects/c8c8fa8f81a74460a0a89c5e78505024 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:17:05.668294 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85740270-b43c-4507-a10c-1cc8162dc2df None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:17:05.668820 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85740270-b43c-4507-a10c-1cc8162dc2df None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:17:05.669102 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85740270-b43c-4507-a10c-1cc8162dc2df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:17:05.695614 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-85740270-b43c-4507-a10c-1cc8162dc2df None None] MFA Rules not processed for user `a7d0a1ee39274a0d90a2be9fbc26a58b`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:17:05.697760 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-85740270-b43c-4507-a10c-1cc8162dc2df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:17:05.715518 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:17:05.779692 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1203/2415] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:17:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:17:07.568377 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91606d45-7b34-4d68-af65-f7def9cc94d6 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:17:07.568993 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91606d45-7b34-4d68-af65-f7def9cc94d6 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:17:07.569398 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91606d45-7b34-4d68-af65-f7def9cc94d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:17:07.595476 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-91606d45-7b34-4d68-af65-f7def9cc94d6 None None] MFA Rules not processed for user `755b04cae45746569bfbf92f2f213a58`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:17:07.596940 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-91606d45-7b34-4d68-af65-f7def9cc94d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:17:07.607680 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:17:07.709965 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1213/2416] 10.222.0.61 () {58 vars in 1009 bytes} [Fri Aug 26 12:17:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:17:07.725913 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85740270-b43c-4507-a10c-1cc8162dc2df None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:17:07.727529 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-85740270-b43c-4507-a10c-1cc8162dc2df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:17:07.835032 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85740270-b43c-4507-a10c-1cc8162dc2df None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:17:07.841386 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c618aab5-f441-49d6-ae77-55f3ac676e7f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead95a244d11432d9fe9f718e1a75f38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:17:07.842006 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c618aab5-f441-49d6-ae77-55f3ac676e7f service swift] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:17:07.842289 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c618aab5-f441-49d6-ae77-55f3ac676e7f service swift] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:17:07.842567 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c618aab5-f441-49d6-ae77-55f3ac676e7f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:17:07.851313 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c618aab5-f441-49d6-ae77-55f3ac676e7f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:17:07.853252 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c618aab5-f441-49d6-ae77-55f3ac676e7f service swift] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:17:07.861921 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1204/2417] 10.222.0.61 () {60 vars in 1384 bytes} [Fri Aug 26 12:17:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 26 12:17:17.699253 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=20 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:17:17.858257 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=49 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:17:20.925171 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91606d45-7b34-4d68-af65-f7def9cc94d6 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:17:20.927081 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-91606d45-7b34-4d68-af65-f7def9cc94d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:17:21.010508 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91606d45-7b34-4d68-af65-f7def9cc94d6 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:17:21.017102 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32be619-2625-4f76-9227-3b8552c838e7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7346e4a29dfb4a0a9768597704be286c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:17:21.017928 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32be619-2625-4f76-9227-3b8552c838e7 service placement] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:17:21.018359 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32be619-2625-4f76-9227-3b8552c838e7 service placement] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:17:21.018793 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32be619-2625-4f76-9227-3b8552c838e7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:17:21.020844 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-a32be619-2625-4f76-9227-3b8552c838e7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:17:21.034192 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32be619-2625-4f76-9227-3b8552c838e7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:17:21.036838 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32be619-2625-4f76-9227-3b8552c838e7 service placement] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:17:21.050407 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1214/2418] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:17:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:17:31.045487 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=32 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:17:45.382051 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c618aab5-f441-49d6-ae77-55f3ac676e7f service swift] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:17:45.383556 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-c618aab5-f441-49d6-ae77-55f3ac676e7f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:17:45.440792 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c618aab5-f441-49d6-ae77-55f3ac676e7f service swift] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:17:45.444368 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5048a6-af37-4d17-8401-e75bd3d71ac3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:17:45.444967 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5048a6-af37-4d17-8401-e75bd3d71ac3 service neutron] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:17:45.445246 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5048a6-af37-4d17-8401-e75bd3d71ac3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:17:45.445520 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5048a6-af37-4d17-8401-e75bd3d71ac3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:17:45.447661 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-ea5048a6-af37-4d17-8401-e75bd3d71ac3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:17:45.510051 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5048a6-af37-4d17-8401-e75bd3d71ac3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:17:45.511473 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5048a6-af37-4d17-8401-e75bd3d71ac3 service neutron] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:17:45.521005 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1205/2419] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:17:45 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:17:55.516798 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=50 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:18:22.786488 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76100d30-8b60-484e-b07d-e6d15da4ecb5 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:18:22.786999 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76100d30-8b60-484e-b07d-e6d15da4ecb5 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:18:22.787089 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76100d30-8b60-484e-b07d-e6d15da4ecb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:18:22.815996 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-76100d30-8b60-484e-b07d-e6d15da4ecb5 None None] MFA Rules not processed for user `a02508c20bd547708638c1875101da6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:18:22.817260 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-76100d30-8b60-484e-b07d-e6d15da4ecb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:18:22.892543 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1215/2420] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:18:22 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:18:32.885000 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=25 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:18:38.864903 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f2650-148e-4e05-9e7f-5b44941ed9d1 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:18:38.865449 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f2650-148e-4e05-9e7f-5b44941ed9d1 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:18:38.865449 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f2650-148e-4e05-9e7f-5b44941ed9d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:18:38.884873 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-0c6f2650-148e-4e05-9e7f-5b44941ed9d1 None None] MFA Rules not processed for user `36a27ca98a2e4521b269be48c9ec4cb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:18:38.886074 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-0c6f2650-148e-4e05-9e7f-5b44941ed9d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:18:38.943845 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1206/2421] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:18:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:18:39.538754 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd118c7-03e0-4c27-bfe7-6a994e228b97 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:18:39.539414 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd118c7-03e0-4c27-bfe7-6a994e228b97 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:18:39.539414 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd118c7-03e0-4c27-bfe7-6a994e228b97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:18:39.564019 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-5fd118c7-03e0-4c27-bfe7-6a994e228b97 None None] MFA Rules not processed for user `a7d0a1ee39274a0d90a2be9fbc26a58b`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:18:39.565374 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-5fd118c7-03e0-4c27-bfe7-6a994e228b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:18:39.581079 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1216/2422] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:18:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:18:48.936577 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=25 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:18:49.553305 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=3 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:19:08.773092 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f2650-148e-4e05-9e7f-5b44941ed9d1 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:19:08.774512 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-0c6f2650-148e-4e05-9e7f-5b44941ed9d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:19:08.834179 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f2650-148e-4e05-9e7f-5b44941ed9d1 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:19:08.837654 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9efec02-d7ff-4c4a-8cc3-559e61512c95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:19:08.838253 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9efec02-d7ff-4c4a-8cc3-559e61512c95 service cinder] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:19:08.838535 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9efec02-d7ff-4c4a-8cc3-559e61512c95 service cinder] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:19:08.838811 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9efec02-d7ff-4c4a-8cc3-559e61512c95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:19:08.841089 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-e9efec02-d7ff-4c4a-8cc3-559e61512c95 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:19:08.910475 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9efec02-d7ff-4c4a-8cc3-559e61512c95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:19:08.911930 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9efec02-d7ff-4c4a-8cc3-559e61512c95 service cinder] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:19:08.922776 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1207/2423] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:19:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:19:10.199923 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd118c7-03e0-4c27-bfe7-6a994e228b97 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:19:10.209705 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd118c7-03e0-4c27-bfe7-6a994e228b97 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:19:10.213355 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b53a95a-6233-406c-b1da-9deefa47eb86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:19:10.213947 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53a95a-6233-406c-b1da-9deefa47eb86 service neutron] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:19:10.214882 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53a95a-6233-406c-b1da-9deefa47eb86 service neutron] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:19:10.215159 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53a95a-6233-406c-b1da-9deefa47eb86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:19:10.217133 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-9b53a95a-6233-406c-b1da-9deefa47eb86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:19:10.231034 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b53a95a-6233-406c-b1da-9deefa47eb86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:19:10.232594 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b53a95a-6233-406c-b1da-9deefa47eb86 service neutron] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:19:10.244483 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1217/2424] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:19:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:19:15.902419 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9efec02-d7ff-4c4a-8cc3-559e61512c95 service cinder] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:19:15.904623 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-e9efec02-d7ff-4c4a-8cc3-559e61512c95 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:19:15.917992 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9efec02-d7ff-4c4a-8cc3-559e61512c95 service cinder] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:19:15.923640 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03229055-b14c-448d-a619-76d76bdd2106 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aacd7d6d17e4f10ba5c9107a12beafa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:19:15.924304 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03229055-b14c-448d-a619-76d76bdd2106 service nova] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:19:15.924579 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03229055-b14c-448d-a619-76d76bdd2106 service nova] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:19:15.924848 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03229055-b14c-448d-a619-76d76bdd2106 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:19:15.926470 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-03229055-b14c-448d-a619-76d76bdd2106 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:19:15.937108 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03229055-b14c-448d-a619-76d76bdd2106 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:19:15.938661 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03229055-b14c-448d-a619-76d76bdd2106 service nova] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:19:15.947719 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1208/2425] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:19:15 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:19:17.111092 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b53a95a-6233-406c-b1da-9deefa47eb86 service neutron] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:19:17.112520 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-9b53a95a-6233-406c-b1da-9deefa47eb86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:19:17.127176 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b53a95a-6233-406c-b1da-9deefa47eb86 service neutron] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:19:17.132706 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203def9e-28c6-4b51-bc3a-2acba58f0440 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aacd7d6d17e4f10ba5c9107a12beafa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:19:17.133297 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203def9e-28c6-4b51-bc3a-2acba58f0440 service nova] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:19:17.133579 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203def9e-28c6-4b51-bc3a-2acba58f0440 service nova] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:19:17.133855 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203def9e-28c6-4b51-bc3a-2acba58f0440 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:19:17.135557 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-203def9e-28c6-4b51-bc3a-2acba58f0440 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:19:17.195216 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203def9e-28c6-4b51-bc3a-2acba58f0440 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:19:17.196816 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203def9e-28c6-4b51-bc3a-2acba58f0440 service nova] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:19:17.206418 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1218/2426] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:19:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:19:21.799555 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bcfb0c-c73f-4a7b-adca-684b3cbac515 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:19:21.799990 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bcfb0c-c73f-4a7b-adca-684b3cbac515 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:19:21.800149 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bcfb0c-c73f-4a7b-adca-684b3cbac515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:19:21.821385 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-d6bcfb0c-c73f-4a7b-adca-684b3cbac515 None None] MFA Rules not processed for user `2d9f1a5e4e77467f8a691b634a841de2`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:19:21.822568 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-d6bcfb0c-c73f-4a7b-adca-684b3cbac515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:19:21.829948 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1209/2427] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:19:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:19:22.137598 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203def9e-28c6-4b51-bc3a-2acba58f0440 service nova] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:19:22.147410 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203def9e-28c6-4b51-bc3a-2acba58f0440 service nova] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:19:22.150954 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea27db92-509f-4045-9921-55e5219c1d4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:19:22.151601 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea27db92-509f-4045-9921-55e5219c1d4a service neutron] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:19:22.151861 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea27db92-509f-4045-9921-55e5219c1d4a service neutron] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:19:22.152132 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea27db92-509f-4045-9921-55e5219c1d4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:19:22.153886 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-ea27db92-509f-4045-9921-55e5219c1d4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:19:22.163186 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea27db92-509f-4045-9921-55e5219c1d4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:19:22.164663 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea27db92-509f-4045-9921-55e5219c1d4a service neutron] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:19:22.173037 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1219/2428] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:19:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:19:22.535405 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bcfb0c-c73f-4a7b-adca-684b3cbac515 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:19:22.555517 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bcfb0c-c73f-4a7b-adca-684b3cbac515 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:19:22.570203 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10dbb08-4789-4751-99ce-93f8e776e392 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7346e4a29dfb4a0a9768597704be286c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:19:22.570980 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10dbb08-4789-4751-99ce-93f8e776e392 service placement] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:19:22.571383 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10dbb08-4789-4751-99ce-93f8e776e392 service placement] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:19:22.571784 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10dbb08-4789-4751-99ce-93f8e776e392 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:19:22.573680 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-e10dbb08-4789-4751-99ce-93f8e776e392 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:19:22.588738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10dbb08-4789-4751-99ce-93f8e776e392 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:19:22.592338 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10dbb08-4789-4751-99ce-93f8e776e392 service placement] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:19:22.607294 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1210/2429] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:19:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:19:32.169045 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=40 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:19:32.600167 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=69 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:19:35.231089 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea27db92-509f-4045-9921-55e5219c1d4a service neutron] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:19:35.232554 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-ea27db92-509f-4045-9921-55e5219c1d4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:19:35.246239 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea27db92-509f-4045-9921-55e5219c1d4a service neutron] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:19:35.250235 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7962bbda-c0ac-4c31-8eea-e384fe4c50de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:19:35.250835 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7962bbda-c0ac-4c31-8eea-e384fe4c50de service neutron] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:19:35.251117 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7962bbda-c0ac-4c31-8eea-e384fe4c50de service neutron] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:19:35.251424 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7962bbda-c0ac-4c31-8eea-e384fe4c50de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:19:35.253131 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-7962bbda-c0ac-4c31-8eea-e384fe4c50de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:19:35.262768 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7962bbda-c0ac-4c31-8eea-e384fe4c50de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:19:35.264142 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7962bbda-c0ac-4c31-8eea-e384fe4c50de service neutron] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:19:35.272555 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1220/2430] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:19:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:19:40.382088 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10dbb08-4789-4751-99ce-93f8e776e392 service placement] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:19:40.399291 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10dbb08-4789-4751-99ce-93f8e776e392 service placement] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:19:40.405753 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d0f4e2-77af-4abd-bdd4-13efd5be842a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aacd7d6d17e4f10ba5c9107a12beafa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:19:40.406405 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d0f4e2-77af-4abd-bdd4-13efd5be842a service nova] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:19:40.406680 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d0f4e2-77af-4abd-bdd4-13efd5be842a service nova] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:19:40.406947 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d0f4e2-77af-4abd-bdd4-13efd5be842a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:19:40.408702 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-48d0f4e2-77af-4abd-bdd4-13efd5be842a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:19:40.422424 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d0f4e2-77af-4abd-bdd4-13efd5be842a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:19:40.424022 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d0f4e2-77af-4abd-bdd4-13efd5be842a service nova] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:19:40.435133 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1211/2431] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:19:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:19:41.608893 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7962bbda-c0ac-4c31-8eea-e384fe4c50de service neutron] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:19:41.617872 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7962bbda-c0ac-4c31-8eea-e384fe4c50de service neutron] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:19:41.621822 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe788a2b-974c-485f-b3ce-7ad57a06da18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:19:41.622538 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe788a2b-974c-485f-b3ce-7ad57a06da18 service neutron] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:19:41.622886 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe788a2b-974c-485f-b3ce-7ad57a06da18 service neutron] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:19:41.623263 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe788a2b-974c-485f-b3ce-7ad57a06da18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:19:41.625958 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-fe788a2b-974c-485f-b3ce-7ad57a06da18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:19:41.696373 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe788a2b-974c-485f-b3ce-7ad57a06da18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:19:41.697809 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe788a2b-974c-485f-b3ce-7ad57a06da18 service neutron] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:19:41.709697 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1221/2432] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:19:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 12:19:42.580901 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d0f4e2-77af-4abd-bdd4-13efd5be842a service nova] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:19:42.604434 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d0f4e2-77af-4abd-bdd4-13efd5be842a service nova] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:19:42.612898 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b481bf55-c2e9-44cf-b66e-79bf6a6172e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aacd7d6d17e4f10ba5c9107a12beafa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:19:42.614267 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b481bf55-c2e9-44cf-b66e-79bf6a6172e4 service nova] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:19:42.614729 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b481bf55-c2e9-44cf-b66e-79bf6a6172e4 service nova] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:19:42.615224 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b481bf55-c2e9-44cf-b66e-79bf6a6172e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:19:42.618779 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-b481bf55-c2e9-44cf-b66e-79bf6a6172e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:19:42.637465 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b481bf55-c2e9-44cf-b66e-79bf6a6172e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:19:42.639518 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b481bf55-c2e9-44cf-b66e-79bf6a6172e4 service nova] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:19:42.673181 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1212/2433] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:19:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:19:45.618371 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe788a2b-974c-485f-b3ce-7ad57a06da18 service neutron] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:19:45.619939 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-fe788a2b-974c-485f-b3ce-7ad57a06da18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:19:45.634057 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe788a2b-974c-485f-b3ce-7ad57a06da18 service neutron] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:19:45.638874 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da10173b-f240-4e3e-a439-80d4988c3e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:19:45.639545 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da10173b-f240-4e3e-a439-80d4988c3e9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:19:45.639830 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da10173b-f240-4e3e-a439-80d4988c3e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:19:45.640133 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da10173b-f240-4e3e-a439-80d4988c3e9e admin admin] PATH_INFO: `/v3/users/a02508c20bd547708638c1875101da6f` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:19:45.641266 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da10173b-f240-4e3e-a439-80d4988c3e9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a02508c20bd547708638c1875101da6f)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:19:45.643252 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da10173b-f240-4e3e-a439-80d4988c3e9e admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:19:45.694092 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-da10173b-f240-4e3e-a439-80d4988c3e9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a02508c20bd547708638c1875101da6f'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:19:45.702094 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-da10173b-f240-4e3e-a439-80d4988c3e9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a02508c20bd547708638c1875101da6f'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:19:45.702833 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-da10173b-f240-4e3e-a439-80d4988c3e9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a02508c20bd547708638c1875101da6f'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:19:45.710423 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-da10173b-f240-4e3e-a439-80d4988c3e9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a02508c20bd547708638c1875101da6f'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:19:45.711083 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-da10173b-f240-4e3e-a439-80d4988c3e9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a02508c20bd547708638c1875101da6f'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:19:45.730271 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1222/2434] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:19:45 2022] DELETE /identity/v3/users/a02508c20bd547708638c1875101da6f => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:19:45.837420 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b481bf55-c2e9-44cf-b66e-79bf6a6172e4 service nova] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:19:45.838908 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-b481bf55-c2e9-44cf-b66e-79bf6a6172e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:19:45.854455 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b481bf55-c2e9-44cf-b66e-79bf6a6172e4 service nova] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:19:45.925456 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379eaa10-2cdf-4d80-a54d-3b4d523c466b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:19:45.926328 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379eaa10-2cdf-4d80-a54d-3b4d523c466b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:19:45.926635 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379eaa10-2cdf-4d80-a54d-3b4d523c466b admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:19:45.927802 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379eaa10-2cdf-4d80-a54d-3b4d523c466b admin admin] PATH_INFO: `/v3/users/a7d0a1ee39274a0d90a2be9fbc26a58b` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:19:45.929701 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379eaa10-2cdf-4d80-a54d-3b4d523c466b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7d0a1ee39274a0d90a2be9fbc26a58b)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:19:45.931728 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379eaa10-2cdf-4d80-a54d-3b4d523c466b admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:19:45.933322 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da10173b-f240-4e3e-a439-80d4988c3e9e admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:19:45.935412 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-da10173b-f240-4e3e-a439-80d4988c3e9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:19:45.951585 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da10173b-f240-4e3e-a439-80d4988c3e9e admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:19:45.956938 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eae31dc-a02e-48a7-93d7-9f421c27e7d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:19:45.957583 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae31dc-a02e-48a7-93d7-9f421c27e7d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:19:45.957866 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae31dc-a02e-48a7-93d7-9f421c27e7d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:19:45.958266 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae31dc-a02e-48a7-93d7-9f421c27e7d0 admin admin] PATH_INFO: `/v3/projects/984b6ea268f04e8989283e6397b68370` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:19:45.959836 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eae31dc-a02e-48a7-93d7-9f421c27e7d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=984b6ea268f04e8989283e6397b68370)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:19:45.961934 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eae31dc-a02e-48a7-93d7-9f421c27e7d0 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:19:45.995154 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-379eaa10-2cdf-4d80-a54d-3b4d523c466b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7d0a1ee39274a0d90a2be9fbc26a58b'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:19:46.003516 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-379eaa10-2cdf-4d80-a54d-3b4d523c466b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7d0a1ee39274a0d90a2be9fbc26a58b'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:19:46.011397 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-379eaa10-2cdf-4d80-a54d-3b4d523c466b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7d0a1ee39274a0d90a2be9fbc26a58b'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:19:46.012056 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-379eaa10-2cdf-4d80-a54d-3b4d523c466b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7d0a1ee39274a0d90a2be9fbc26a58b'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:19:46.015306 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:19:46.016854 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-9eae31dc-a02e-48a7-93d7-9f421c27e7d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '984b6ea268f04e8989283e6397b68370', 'request_id': 'req-9eae31dc-a02e-48a7-93d7-9f421c27e7d0'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:19:46.025251 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-9eae31dc-a02e-48a7-93d7-9f421c27e7d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '984b6ea268f04e8989283e6397b68370', 'request_id': 'req-9eae31dc-a02e-48a7-93d7-9f421c27e7d0'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:19:46.026189 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-379eaa10-2cdf-4d80-a54d-3b4d523c466b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7d0a1ee39274a0d90a2be9fbc26a58b'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:19:46.032584 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1213/2435] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:19:45 2022] DELETE /identity/v3/users/a7d0a1ee39274a0d90a2be9fbc26a58b => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:19:46.033392 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-9eae31dc-a02e-48a7-93d7-9f421c27e7d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '984b6ea268f04e8989283e6397b68370', 'request_id': 'req-9eae31dc-a02e-48a7-93d7-9f421c27e7d0'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:19:46.044523 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-9eae31dc-a02e-48a7-93d7-9f421c27e7d0 admin admin] The token cache is being invalidate because project 984b6ea268f04e8989283e6397b68370 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81712) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:19:46.044818 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-9eae31dc-a02e-48a7-93d7-9f421c27e7d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:19:46.045462 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-9eae31dc-a02e-48a7-93d7-9f421c27e7d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:19:46.047057 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1223/2436] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:19:45 2022] DELETE /identity/v3/projects/984b6ea268f04e8989283e6397b68370 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 12:19:46.052177 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356f41a4-96ed-4120-af1d-49a104c1f6e6 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:19:46.052465 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356f41a4-96ed-4120-af1d-49a104c1f6e6 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:19:46.052743 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356f41a4-96ed-4120-af1d-49a104c1f6e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:19:46.073596 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-356f41a4-96ed-4120-af1d-49a104c1f6e6 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:19:46.074994 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-356f41a4-96ed-4120-af1d-49a104c1f6e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:19:46.126642 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:19:46.141756 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1214/2437] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:19:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:19:46.149547 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59848447-c8a9-4bac-95e3-43a0981b80b8 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:19:46.149839 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59848447-c8a9-4bac-95e3-43a0981b80b8 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:19:46.150122 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59848447-c8a9-4bac-95e3-43a0981b80b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:19:46.170321 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-59848447-c8a9-4bac-95e3-43a0981b80b8 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:19:46.172198 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-59848447-c8a9-4bac-95e3-43a0981b80b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:19:46.180380 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1224/2438] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:19:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:19:46.184671 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356f41a4-96ed-4120-af1d-49a104c1f6e6 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:19:46.196378 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356f41a4-96ed-4120-af1d-49a104c1f6e6 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:19:46.202439 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36740400-346f-4298-9c36-7981c53390b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:19:46.203453 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36740400-346f-4298-9c36-7981c53390b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:19:46.203807 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36740400-346f-4298-9c36-7981c53390b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:19:46.204155 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36740400-346f-4298-9c36-7981c53390b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:19:46.204881 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36740400-346f-4298-9c36-7981c53390b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:19:46.205259 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36740400-346f-4298-9c36-7981c53390b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:19:46.207008 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36740400-346f-4298-9c36-7981c53390b2 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:19:46.210706 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59848447-c8a9-4bac-95e3-43a0981b80b8 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:19:46.216263 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-59848447-c8a9-4bac-95e3-43a0981b80b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:19:46.220880 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1215/2439] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:19:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:19:46.224473 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36740400-346f-4298-9c36-7981c53390b2 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:19:46.229842 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59848447-c8a9-4bac-95e3-43a0981b80b8 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:19:46.235679 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101f47f-0f48-4ae2-8509-8d1b8fb80f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:19:46.236318 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101f47f-0f48-4ae2-8509-8d1b8fb80f41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:19:46.236596 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101f47f-0f48-4ae2-8509-8d1b8fb80f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:19:46.236869 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101f47f-0f48-4ae2-8509-8d1b8fb80f41 admin admin] PATH_INFO: `/v3/projects/2ab22ea009914891b1f594ed3d1e4ec3` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:19:46.237997 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4101f47f-0f48-4ae2-8509-8d1b8fb80f41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ab22ea009914891b1f594ed3d1e4ec3)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:19:46.239249 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36740400-346f-4298-9c36-7981c53390b2 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:19:46.240282 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4101f47f-0f48-4ae2-8509-8d1b8fb80f41 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:19:46.245593 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310a0c9d-9ad4-4bc5-82a4-58a4fc0c0965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:19:46.246083 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a0c9d-9ad4-4bc5-82a4-58a4fc0c0965 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:19:46.246398 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a0c9d-9ad4-4bc5-82a4-58a4fc0c0965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:19:46.246965 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a0c9d-9ad4-4bc5-82a4-58a4fc0c0965 admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:19:46.247795 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310a0c9d-9ad4-4bc5-82a4-58a4fc0c0965 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:19:46.250618 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310a0c9d-9ad4-4bc5-82a4-58a4fc0c0965 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:19:46.281533 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1216/2440] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:19:46 2022] POST /identity/v3/projects => generated 389 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 12:19:46.286435 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310a0c9d-9ad4-4bc5-82a4-58a4fc0c0965 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:19:46.292888 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-4101f47f-0f48-4ae2-8509-8d1b8fb80f41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ab22ea009914891b1f594ed3d1e4ec3', 'request_id': 'req-4101f47f-0f48-4ae2-8509-8d1b8fb80f41'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:19:46.299578 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-4101f47f-0f48-4ae2-8509-8d1b8fb80f41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ab22ea009914891b1f594ed3d1e4ec3', 'request_id': 'req-4101f47f-0f48-4ae2-8509-8d1b8fb80f41'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:19:46.307691 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-4101f47f-0f48-4ae2-8509-8d1b8fb80f41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ab22ea009914891b1f594ed3d1e4ec3', 'request_id': 'req-4101f47f-0f48-4ae2-8509-8d1b8fb80f41'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:19:46.318442 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-4101f47f-0f48-4ae2-8509-8d1b8fb80f41 admin admin] The token cache is being invalidate because project 2ab22ea009914891b1f594ed3d1e4ec3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81712) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:19:46.318751 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-4101f47f-0f48-4ae2-8509-8d1b8fb80f41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:19:46.319421 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-4101f47f-0f48-4ae2-8509-8d1b8fb80f41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:19:46.320948 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1225/2441] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:19:46 2022] DELETE /identity/v3/projects/2ab22ea009914891b1f594ed3d1e4ec3 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:19:46.353101 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310a0c9d-9ad4-4bc5-82a4-58a4fc0c0965 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:19:46.358058 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7297959-4021-49bf-af7b-6aaa60d32037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:19:46.359323 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7297959-4021-49bf-af7b-6aaa60d32037 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:19:46.359617 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7297959-4021-49bf-af7b-6aaa60d32037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:19:46.359888 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7297959-4021-49bf-af7b-6aaa60d32037 admin admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:19:46.360532 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7297959-4021-49bf-af7b-6aaa60d32037 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:19:46.362516 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7297959-4021-49bf-af7b-6aaa60d32037 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:19:46.386074 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1217/2442] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:19:46 2022] POST /identity/v3/users => generated 295 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 12:19:46.390963 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101f47f-0f48-4ae2-8509-8d1b8fb80f41 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:19:46.392606 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-4101f47f-0f48-4ae2-8509-8d1b8fb80f41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:19:46.465196 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101f47f-0f48-4ae2-8509-8d1b8fb80f41 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:19:46.470914 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3969fa-1211-4695-98f1-7b7a910d1216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:19:46.471576 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3969fa-1211-4695-98f1-7b7a910d1216 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:19:46.471853 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3969fa-1211-4695-98f1-7b7a910d1216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:19:46.472124 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3969fa-1211-4695-98f1-7b7a910d1216 admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:19:46.472775 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3969fa-1211-4695-98f1-7b7a910d1216 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:19:46.474409 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3969fa-1211-4695-98f1-7b7a910d1216 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:19:46.485521 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1226/2443] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:19:46 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:19:46.490044 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7297959-4021-49bf-af7b-6aaa60d32037 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:19:46.501182 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7297959-4021-49bf-af7b-6aaa60d32037 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:19:46.506015 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b3c803-ecda-4e84-9643-addd8e42eb5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:19:46.506744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b3c803-ecda-4e84-9643-addd8e42eb5c admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:19:46.507091 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b3c803-ecda-4e84-9643-addd8e42eb5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:19:46.507487 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b3c803-ecda-4e84-9643-addd8e42eb5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:19:46.508526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b3c803-ecda-4e84-9643-addd8e42eb5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:19:46.510714 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b3c803-ecda-4e84-9643-addd8e42eb5c admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:19:46.522156 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1218/2444] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:19:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:19:46.526490 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3969fa-1211-4695-98f1-7b7a910d1216 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:19:46.537108 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3969fa-1211-4695-98f1-7b7a910d1216 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:19:46.541693 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54e5e73-5845-48ee-bc6d-30703b784f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:19:46.542331 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e5e73-5845-48ee-bc6d-30703b784f0b admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:19:46.542611 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e5e73-5845-48ee-bc6d-30703b784f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:19:46.542885 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e5e73-5845-48ee-bc6d-30703b784f0b admin admin] PATH_INFO: `/v3/projects/f4583d65a595463b97e8dfabbf3da521/users/0e013faf32b84179989eb99be958555f/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:19:46.558459 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54e5e73-5845-48ee-bc6d-30703b784f0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4583d65a595463b97e8dfabbf3da521, user_id=0e013faf32b84179989eb99be958555f, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:19:46.561118 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54e5e73-5845-48ee-bc6d-30703b784f0b admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:19:46.573487 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1227/2445] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:19:46 2022] PUT /identity/v3/projects/f4583d65a595463b97e8dfabbf3da521/users/0e013faf32b84179989eb99be958555f/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:19:46.578972 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d874451c-4aa8-4d18-adda-88f974a47e9d None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:19:46.579296 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d874451c-4aa8-4d18-adda-88f974a47e9d None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:19:46.579580 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d874451c-4aa8-4d18-adda-88f974a47e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:19:46.599095 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-d874451c-4aa8-4d18-adda-88f974a47e9d None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:19:46.600307 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-d874451c-4aa8-4d18-adda-88f974a47e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:19:46.659056 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1219/2446] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:19:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:19:46.668546 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54e5e73-5845-48ee-bc6d-30703b784f0b admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:19:46.670249 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-a54e5e73-5845-48ee-bc6d-30703b784f0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:19:46.698473 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:19:46.729362 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54e5e73-5845-48ee-bc6d-30703b784f0b admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:19:46.734325 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60512a16-d80b-4237-9adc-65dbe1d6d8f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:19:46.734916 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60512a16-d80b-4237-9adc-65dbe1d6d8f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:19:46.735213 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60512a16-d80b-4237-9adc-65dbe1d6d8f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:19:46.735502 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60512a16-d80b-4237-9adc-65dbe1d6d8f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:19:46.748220 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60512a16-d80b-4237-9adc-65dbe1d6d8f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:19:46.750453 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60512a16-d80b-4237-9adc-65dbe1d6d8f9 service neutron] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:19:46.760214 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1228/2447] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:19:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:19:48.696474 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d874451c-4aa8-4d18-adda-88f974a47e9d None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:19:48.697938 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-d874451c-4aa8-4d18-adda-88f974a47e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:19:48.751753 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:19:48.789310 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d874451c-4aa8-4d18-adda-88f974a47e9d None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:19:48.802105 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212f5989-acb3-45cc-a194-48fe2795772d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aacd7d6d17e4f10ba5c9107a12beafa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:19:48.803154 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f5989-acb3-45cc-a194-48fe2795772d service nova] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:19:48.803608 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f5989-acb3-45cc-a194-48fe2795772d service nova] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:19:48.803987 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f5989-acb3-45cc-a194-48fe2795772d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:19:48.806313 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-212f5989-acb3-45cc-a194-48fe2795772d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:19:48.902744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212f5989-acb3-45cc-a194-48fe2795772d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:19:48.905265 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212f5989-acb3-45cc-a194-48fe2795772d service nova] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:19:48.919098 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1220/2448] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:19:48 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:19:50.559168 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579dac0b-77b9-483e-9648-35f8613d8498 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:19:50.560026 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579dac0b-77b9-483e-9648-35f8613d8498 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:19:50.560505 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579dac0b-77b9-483e-9648-35f8613d8498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:19:50.596858 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-579dac0b-77b9-483e-9648-35f8613d8498 None None] MFA Rules not processed for user `0e013faf32b84179989eb99be958555f`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:19:50.598841 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-579dac0b-77b9-483e-9648-35f8613d8498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:19:50.692490 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1229/2449] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:19:50 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:19:50.711332 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212f5989-acb3-45cc-a194-48fe2795772d service nova] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:19:50.734994 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212f5989-acb3-45cc-a194-48fe2795772d service nova] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:19:50.748347 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604618e4-dd72-4ef7-b404-96979bb2335f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aacd7d6d17e4f10ba5c9107a12beafa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:19:50.752824 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604618e4-dd72-4ef7-b404-96979bb2335f service nova] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:19:50.756188 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604618e4-dd72-4ef7-b404-96979bb2335f service nova] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:19:50.756188 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604618e4-dd72-4ef7-b404-96979bb2335f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:19:50.768317 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604618e4-dd72-4ef7-b404-96979bb2335f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:19:50.770223 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604618e4-dd72-4ef7-b404-96979bb2335f service nova] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:19:50.784549 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1221/2450] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:19:50 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:19:52.360425 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6096662b-b57c-424f-9b0b-b46384bbb264 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:19:52.360898 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6096662b-b57c-424f-9b0b-b46384bbb264 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:19:52.361014 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6096662b-b57c-424f-9b0b-b46384bbb264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:19:52.381279 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-6096662b-b57c-424f-9b0b-b46384bbb264 None None] MFA Rules not processed for user `0e013faf32b84179989eb99be958555f`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:19:52.382505 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-6096662b-b57c-424f-9b0b-b46384bbb264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:19:52.389456 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1230/2451] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:19:52 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:19:52.703126 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604618e4-dd72-4ef7-b404-96979bb2335f service nova] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:19:52.705266 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-604618e4-dd72-4ef7-b404-96979bb2335f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:19:52.717392 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604618e4-dd72-4ef7-b404-96979bb2335f service nova] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:19:52.721144 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b625ef4-49cb-4585-bd40-7983f3d9b364 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:19:52.721742 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b625ef4-49cb-4585-bd40-7983f3d9b364 service neutron] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:19:52.722018 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b625ef4-49cb-4585-bd40-7983f3d9b364 service neutron] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:19:52.722286 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b625ef4-49cb-4585-bd40-7983f3d9b364 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:19:52.723993 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-4b625ef4-49cb-4585-bd40-7983f3d9b364 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:19:52.733803 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b625ef4-49cb-4585-bd40-7983f3d9b364 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:19:52.735289 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b625ef4-49cb-4585-bd40-7983f3d9b364 service neutron] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:19:52.743809 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1222/2452] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:19:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:20:01.028548 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6096662b-b57c-424f-9b0b-b46384bbb264 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:20:01.042674 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6096662b-b57c-424f-9b0b-b46384bbb264 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:20:01.050901 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ec45f8-b422-4d8f-b1b3-fde7f2e7644c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aacd7d6d17e4f10ba5c9107a12beafa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:20:01.051576 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec45f8-b422-4d8f-b1b3-fde7f2e7644c service nova] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:01.051851 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec45f8-b422-4d8f-b1b3-fde7f2e7644c service nova] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:01.052640 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec45f8-b422-4d8f-b1b3-fde7f2e7644c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:01.055136 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-89ec45f8-b422-4d8f-b1b3-fde7f2e7644c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:20:01.069968 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ec45f8-b422-4d8f-b1b3-fde7f2e7644c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:20:01.071998 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ec45f8-b422-4d8f-b1b3-fde7f2e7644c service nova] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:20:01.086112 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1231/2453] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:20:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:20:02.740549 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=51 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:20:08.782293 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d18f73-4467-468b-bd64-c7cf687f034b None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:08.782923 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d18f73-4467-468b-bd64-c7cf687f034b None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:08.782923 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d18f73-4467-468b-bd64-c7cf687f034b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:08.803645 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-f7d18f73-4467-468b-bd64-c7cf687f034b None None] MFA Rules not processed for user `36a27ca98a2e4521b269be48c9ec4cb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:20:08.805201 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-f7d18f73-4467-468b-bd64-c7cf687f034b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:20:08.820190 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1223/2454] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:20:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:20:10.316748 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a9eded-eb6b-47d9-83e4-a2f8eb26aea0 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:10.317240 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a9eded-eb6b-47d9-83e4-a2f8eb26aea0 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:10.317335 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a9eded-eb6b-47d9-83e4-a2f8eb26aea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:10.337167 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-49a9eded-eb6b-47d9-83e4-a2f8eb26aea0 None None] MFA Rules not processed for user `755b04cae45746569bfbf92f2f213a58`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:20:10.338595 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-49a9eded-eb6b-47d9-83e4-a2f8eb26aea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:20:10.394989 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1232/2455] 10.222.0.61 () {58 vars in 967 bytes} [Fri Aug 26 12:20:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:20:10.435438 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d18f73-4467-468b-bd64-c7cf687f034b None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:20:10.437532 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-f7d18f73-4467-468b-bd64-c7cf687f034b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:20:10.506376 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d18f73-4467-468b-bd64-c7cf687f034b None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:20:10.511276 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0ad17c-c0d5-4627-94b5-ec6c68d01b6b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead95a244d11432d9fe9f718e1a75f38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:20:10.511877 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ad17c-c0d5-4627-94b5-ec6c68d01b6b service swift] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:10.512157 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ad17c-c0d5-4627-94b5-ec6c68d01b6b service swift] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:10.512429 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ad17c-c0d5-4627-94b5-ec6c68d01b6b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:10.520380 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0ad17c-c0d5-4627-94b5-ec6c68d01b6b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:20:10.521951 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0ad17c-c0d5-4627-94b5-ec6c68d01b6b service swift] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:20:10.530120 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1224/2456] 10.222.0.61 () {60 vars in 1384 bytes} [Fri Aug 26 12:20:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 12:20:17.889739 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a9eded-eb6b-47d9-83e4-a2f8eb26aea0 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:20:17.891514 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-49a9eded-eb6b-47d9-83e4-a2f8eb26aea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:20:17.902574 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=82 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:20:17.908311 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a9eded-eb6b-47d9-83e4-a2f8eb26aea0 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:20:17.913542 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4422724-69cd-48ec-a2cf-452c27583f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:20:17.914185 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4422724-69cd-48ec-a2cf-452c27583f59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:17.915005 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4422724-69cd-48ec-a2cf-452c27583f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:17.915005 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4422724-69cd-48ec-a2cf-452c27583f59 admin admin] PATH_INFO: `/v3/users/ceed3909469f409189d9ed93937003f0` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:17.916004 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4422724-69cd-48ec-a2cf-452c27583f59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ceed3909469f409189d9ed93937003f0)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:20:17.917952 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4422724-69cd-48ec-a2cf-452c27583f59 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:20:17.968777 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-e4422724-69cd-48ec-a2cf-452c27583f59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ceed3909469f409189d9ed93937003f0'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:17.976817 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-e4422724-69cd-48ec-a2cf-452c27583f59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ceed3909469f409189d9ed93937003f0'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:17.979825 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-e4422724-69cd-48ec-a2cf-452c27583f59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ceed3909469f409189d9ed93937003f0'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:17.990241 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-e4422724-69cd-48ec-a2cf-452c27583f59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ceed3909469f409189d9ed93937003f0'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:17.990913 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-e4422724-69cd-48ec-a2cf-452c27583f59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ceed3909469f409189d9ed93937003f0'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:18.008826 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1233/2457] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:20:17 2022] DELETE /identity/v3/users/ceed3909469f409189d9ed93937003f0 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:20:18.210330 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0ad17c-c0d5-4627-94b5-ec6c68d01b6b service swift] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:20:18.221549 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0ad17c-c0d5-4627-94b5-ec6c68d01b6b service swift] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:20:18.291249 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c050e48-3821-46de-9b3f-5330efd80a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:20:18.291887 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c050e48-3821-46de-9b3f-5330efd80a2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:18.292169 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c050e48-3821-46de-9b3f-5330efd80a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:18.292440 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c050e48-3821-46de-9b3f-5330efd80a2b admin admin] PATH_INFO: `/v3/projects/1f876d0f217a485392ddcbd79134f609` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:18.293550 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c050e48-3821-46de-9b3f-5330efd80a2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f876d0f217a485392ddcbd79134f609)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:20:18.295994 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c050e48-3821-46de-9b3f-5330efd80a2b admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:20:18.344326 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-3c050e48-3821-46de-9b3f-5330efd80a2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f876d0f217a485392ddcbd79134f609', 'request_id': 'req-3c050e48-3821-46de-9b3f-5330efd80a2b'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:18.350226 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-3c050e48-3821-46de-9b3f-5330efd80a2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f876d0f217a485392ddcbd79134f609', 'request_id': 'req-3c050e48-3821-46de-9b3f-5330efd80a2b'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:18.357184 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-3c050e48-3821-46de-9b3f-5330efd80a2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f876d0f217a485392ddcbd79134f609', 'request_id': 'req-3c050e48-3821-46de-9b3f-5330efd80a2b'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:18.368157 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-3c050e48-3821-46de-9b3f-5330efd80a2b admin admin] The token cache is being invalidate because project 1f876d0f217a485392ddcbd79134f609 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:20:18.368442 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-3c050e48-3821-46de-9b3f-5330efd80a2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:18.369051 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-3c050e48-3821-46de-9b3f-5330efd80a2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:18.370744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1225/2458] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:20:18 2022] DELETE /identity/v3/projects/1f876d0f217a485392ddcbd79134f609 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:20:18.374787 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4422724-69cd-48ec-a2cf-452c27583f59 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:20:18.376286 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-e4422724-69cd-48ec-a2cf-452c27583f59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:20:18.389348 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4422724-69cd-48ec-a2cf-452c27583f59 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:20:18.447312 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06b7414-9ce0-47c1-afe9-f7ea5de1e9e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:20:18.447977 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06b7414-9ce0-47c1-afe9-f7ea5de1e9e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:18.448255 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06b7414-9ce0-47c1-afe9-f7ea5de1e9e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:18.448525 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06b7414-9ce0-47c1-afe9-f7ea5de1e9e1 admin admin] PATH_INFO: `/v3/users/2e38dc9b4d0c4308a355442328911615` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:18.449637 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06b7414-9ce0-47c1-afe9-f7ea5de1e9e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e38dc9b4d0c4308a355442328911615)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:20:18.451692 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06b7414-9ce0-47c1-afe9-f7ea5de1e9e1 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:20:18.496593 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-d06b7414-9ce0-47c1-afe9-f7ea5de1e9e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e38dc9b4d0c4308a355442328911615'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:18.503601 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-d06b7414-9ce0-47c1-afe9-f7ea5de1e9e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e38dc9b4d0c4308a355442328911615'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:18.504197 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-d06b7414-9ce0-47c1-afe9-f7ea5de1e9e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e38dc9b4d0c4308a355442328911615'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:18.511662 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-d06b7414-9ce0-47c1-afe9-f7ea5de1e9e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e38dc9b4d0c4308a355442328911615'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:18.512330 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-d06b7414-9ce0-47c1-afe9-f7ea5de1e9e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e38dc9b4d0c4308a355442328911615'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:18.523002 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:20:18.533106 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1234/2459] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:20:18 2022] DELETE /identity/v3/users/2e38dc9b4d0c4308a355442328911615 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:20:18.699411 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c050e48-3821-46de-9b3f-5330efd80a2b admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:20:18.700842 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-3c050e48-3821-46de-9b3f-5330efd80a2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:20:18.716513 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c050e48-3821-46de-9b3f-5330efd80a2b admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:20:18.727078 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:20:18.777054 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8517c4d1-7356-41ab-9212-af6494db7396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:20:18.777678 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8517c4d1-7356-41ab-9212-af6494db7396 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:18.777958 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8517c4d1-7356-41ab-9212-af6494db7396 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:18.778231 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8517c4d1-7356-41ab-9212-af6494db7396 admin admin] PATH_INFO: `/v3/projects/44cb1c468c464288bd570487f79770fd` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:18.779361 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8517c4d1-7356-41ab-9212-af6494db7396 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44cb1c468c464288bd570487f79770fd)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:20:18.781881 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8517c4d1-7356-41ab-9212-af6494db7396 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:20:18.827824 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-8517c4d1-7356-41ab-9212-af6494db7396 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44cb1c468c464288bd570487f79770fd', 'request_id': 'req-8517c4d1-7356-41ab-9212-af6494db7396'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:18.834822 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-8517c4d1-7356-41ab-9212-af6494db7396 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44cb1c468c464288bd570487f79770fd', 'request_id': 'req-8517c4d1-7356-41ab-9212-af6494db7396'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:18.844408 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-8517c4d1-7356-41ab-9212-af6494db7396 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44cb1c468c464288bd570487f79770fd', 'request_id': 'req-8517c4d1-7356-41ab-9212-af6494db7396'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:18.855096 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-8517c4d1-7356-41ab-9212-af6494db7396 admin admin] The token cache is being invalidate because project 44cb1c468c464288bd570487f79770fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:20:18.855414 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-8517c4d1-7356-41ab-9212-af6494db7396 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:18.856041 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-8517c4d1-7356-41ab-9212-af6494db7396 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:18.857512 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1226/2460] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:20:18 2022] DELETE /identity/v3/projects/44cb1c468c464288bd570487f79770fd => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:20:28.534310 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:DELETE=1,keystone:SELECT=2 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:20:28.852074 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=45,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:20:38.707100 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06b7414-9ce0-47c1-afe9-f7ea5de1e9e1 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:20:38.709250 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-d06b7414-9ce0-47c1-afe9-f7ea5de1e9e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:20:38.800950 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8517c4d1-7356-41ab-9212-af6494db7396 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:20:38.803551 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-8517c4d1-7356-41ab-9212-af6494db7396 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:20:38.845159 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06b7414-9ce0-47c1-afe9-f7ea5de1e9e1 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:20:38.902464 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8517c4d1-7356-41ab-9212-af6494db7396 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:20:38.909316 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555f3d87-49e7-4754-b4f7-e2885b286cc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:20:38.910099 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555f3d87-49e7-4754-b4f7-e2885b286cc1 service neutron] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:38.910570 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555f3d87-49e7-4754-b4f7-e2885b286cc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:38.911045 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555f3d87-49e7-4754-b4f7-e2885b286cc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:38.913560 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-555f3d87-49e7-4754-b4f7-e2885b286cc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:20:38.919918 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53be3556-a9b6-446b-ac49-b4733aae08b4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7346e4a29dfb4a0a9768597704be286c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:20:38.922092 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53be3556-a9b6-446b-ac49-b4733aae08b4 service placement] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:38.922308 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53be3556-a9b6-446b-ac49-b4733aae08b4 service placement] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:38.922486 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53be3556-a9b6-446b-ac49-b4733aae08b4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:38.926849 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-53be3556-a9b6-446b-ac49-b4733aae08b4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:20:38.941889 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53be3556-a9b6-446b-ac49-b4733aae08b4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:20:38.943853 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53be3556-a9b6-446b-ac49-b4733aae08b4 service placement] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:20:38.960083 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1235/2461] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:20:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:20:39.012736 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555f3d87-49e7-4754-b4f7-e2885b286cc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:20:39.014177 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555f3d87-49e7-4754-b4f7-e2885b286cc1 service neutron] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:20:39.027638 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1227/2462] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:20:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:20:42.271417 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53be3556-a9b6-446b-ac49-b4733aae08b4 service placement] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:20:42.273596 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-53be3556-a9b6-446b-ac49-b4733aae08b4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:20:42.289236 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53be3556-a9b6-446b-ac49-b4733aae08b4 service placement] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:20:42.294040 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21dd625-70ec-40f8-8ea2-62e049ffcf22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:20:42.294679 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21dd625-70ec-40f8-8ea2-62e049ffcf22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:42.294955 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21dd625-70ec-40f8-8ea2-62e049ffcf22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:42.295431 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21dd625-70ec-40f8-8ea2-62e049ffcf22 admin admin] PATH_INFO: `/v3/users/2b51cd55973a419cad6311609395bb89` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:42.298006 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21dd625-70ec-40f8-8ea2-62e049ffcf22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b51cd55973a419cad6311609395bb89)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:20:42.300368 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21dd625-70ec-40f8-8ea2-62e049ffcf22 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:20:42.349281 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-f21dd625-70ec-40f8-8ea2-62e049ffcf22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b51cd55973a419cad6311609395bb89'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:42.356658 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-f21dd625-70ec-40f8-8ea2-62e049ffcf22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b51cd55973a419cad6311609395bb89'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:42.357261 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-f21dd625-70ec-40f8-8ea2-62e049ffcf22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b51cd55973a419cad6311609395bb89'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:42.364997 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-f21dd625-70ec-40f8-8ea2-62e049ffcf22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b51cd55973a419cad6311609395bb89'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:42.365619 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-f21dd625-70ec-40f8-8ea2-62e049ffcf22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b51cd55973a419cad6311609395bb89'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:42.382669 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1236/2463] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:20:42 2022] DELETE /identity/v3/users/2b51cd55973a419cad6311609395bb89 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:20:42.548717 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555f3d87-49e7-4754-b4f7-e2885b286cc1 service neutron] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:20:42.558992 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555f3d87-49e7-4754-b4f7-e2885b286cc1 service neutron] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:20:42.622436 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455fe55e-0e67-49ce-aa0b-480716b6c95d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:20:42.623072 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455fe55e-0e67-49ce-aa0b-480716b6c95d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:42.623381 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455fe55e-0e67-49ce-aa0b-480716b6c95d admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:42.623652 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455fe55e-0e67-49ce-aa0b-480716b6c95d admin admin] PATH_INFO: `/v3/projects/293ca4bc65af4f25afb8456f32378173` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:42.624785 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455fe55e-0e67-49ce-aa0b-480716b6c95d admin admin] RBAC: Authorizing `identity:delete_project(project_id=293ca4bc65af4f25afb8456f32378173)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:20:42.626672 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455fe55e-0e67-49ce-aa0b-480716b6c95d admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:20:42.672735 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-455fe55e-0e67-49ce-aa0b-480716b6c95d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '293ca4bc65af4f25afb8456f32378173', 'request_id': 'req-455fe55e-0e67-49ce-aa0b-480716b6c95d'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:42.674595 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:20:42.679695 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-455fe55e-0e67-49ce-aa0b-480716b6c95d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '293ca4bc65af4f25afb8456f32378173', 'request_id': 'req-455fe55e-0e67-49ce-aa0b-480716b6c95d'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:42.688241 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-455fe55e-0e67-49ce-aa0b-480716b6c95d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '293ca4bc65af4f25afb8456f32378173', 'request_id': 'req-455fe55e-0e67-49ce-aa0b-480716b6c95d'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:42.698285 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-455fe55e-0e67-49ce-aa0b-480716b6c95d admin admin] The token cache is being invalidate because project 293ca4bc65af4f25afb8456f32378173 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:20:42.698598 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-455fe55e-0e67-49ce-aa0b-480716b6c95d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:42.699223 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-455fe55e-0e67-49ce-aa0b-480716b6c95d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:42.700698 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1228/2464] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:20:42 2022] DELETE /identity/v3/projects/293ca4bc65af4f25afb8456f32378173 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:20:42.705333 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21dd625-70ec-40f8-8ea2-62e049ffcf22 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:20:42.706709 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-f21dd625-70ec-40f8-8ea2-62e049ffcf22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:20:42.715910 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:20:42.720571 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21dd625-70ec-40f8-8ea2-62e049ffcf22 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:20:42.787517 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dd22c6-6982-4198-a4c0-dbe50312dd6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:20:42.788147 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd22c6-6982-4198-a4c0-dbe50312dd6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:42.788427 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd22c6-6982-4198-a4c0-dbe50312dd6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:42.788697 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd22c6-6982-4198-a4c0-dbe50312dd6b admin admin] PATH_INFO: `/v3/users/194922a3d1974964997c4905dcf85359` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:42.789837 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dd22c6-6982-4198-a4c0-dbe50312dd6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=194922a3d1974964997c4905dcf85359)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:20:42.791795 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dd22c6-6982-4198-a4c0-dbe50312dd6b admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:20:42.840283 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-02dd22c6-6982-4198-a4c0-dbe50312dd6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '194922a3d1974964997c4905dcf85359'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:42.848874 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-02dd22c6-6982-4198-a4c0-dbe50312dd6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '194922a3d1974964997c4905dcf85359'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:42.849651 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-02dd22c6-6982-4198-a4c0-dbe50312dd6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '194922a3d1974964997c4905dcf85359'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:42.857163 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-02dd22c6-6982-4198-a4c0-dbe50312dd6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '194922a3d1974964997c4905dcf85359'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:42.857688 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-02dd22c6-6982-4198-a4c0-dbe50312dd6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '194922a3d1974964997c4905dcf85359'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:42.878476 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1237/2465] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:20:42 2022] DELETE /identity/v3/users/194922a3d1974964997c4905dcf85359 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:20:43.055449 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455fe55e-0e67-49ce-aa0b-480716b6c95d admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:20:43.057312 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-455fe55e-0e67-49ce-aa0b-480716b6c95d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:20:43.072852 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455fe55e-0e67-49ce-aa0b-480716b6c95d admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:20:43.133581 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f39e292-cfab-450a-9d4d-cb5bac859d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:20:43.134213 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f39e292-cfab-450a-9d4d-cb5bac859d2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:43.134491 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f39e292-cfab-450a-9d4d-cb5bac859d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:43.134895 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f39e292-cfab-450a-9d4d-cb5bac859d2c admin admin] PATH_INFO: `/v3/projects/bf6dc52acadd44a493ed8e386e803a99` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:43.136298 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f39e292-cfab-450a-9d4d-cb5bac859d2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf6dc52acadd44a493ed8e386e803a99)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:20:43.138739 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f39e292-cfab-450a-9d4d-cb5bac859d2c admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:20:43.188578 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-0f39e292-cfab-450a-9d4d-cb5bac859d2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf6dc52acadd44a493ed8e386e803a99', 'request_id': 'req-0f39e292-cfab-450a-9d4d-cb5bac859d2c'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:43.194578 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-0f39e292-cfab-450a-9d4d-cb5bac859d2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf6dc52acadd44a493ed8e386e803a99', 'request_id': 'req-0f39e292-cfab-450a-9d4d-cb5bac859d2c'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:43.202473 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-0f39e292-cfab-450a-9d4d-cb5bac859d2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf6dc52acadd44a493ed8e386e803a99', 'request_id': 'req-0f39e292-cfab-450a-9d4d-cb5bac859d2c'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:43.214732 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-0f39e292-cfab-450a-9d4d-cb5bac859d2c admin admin] The token cache is being invalidate because project bf6dc52acadd44a493ed8e386e803a99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:20:43.215019 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-0f39e292-cfab-450a-9d4d-cb5bac859d2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:43.215819 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-0f39e292-cfab-450a-9d4d-cb5bac859d2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:43.217283 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1229/2466] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:20:43 2022] DELETE /identity/v3/projects/bf6dc52acadd44a493ed8e386e803a99 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 12:20:48.642205 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dd22c6-6982-4198-a4c0-dbe50312dd6b admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:20:48.643692 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-02dd22c6-6982-4198-a4c0-dbe50312dd6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:20:48.699862 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dd22c6-6982-4198-a4c0-dbe50312dd6b admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:20:48.737217 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:20:48.755596 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76469d1f-2b32-44a9-9b23-510f7279d3c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:20:48.756206 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76469d1f-2b32-44a9-9b23-510f7279d3c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:48.756481 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76469d1f-2b32-44a9-9b23-510f7279d3c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:48.756751 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76469d1f-2b32-44a9-9b23-510f7279d3c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:48.758486 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-76469d1f-2b32-44a9-9b23-510f7279d3c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:20:48.822596 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76469d1f-2b32-44a9-9b23-510f7279d3c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:20:48.824224 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76469d1f-2b32-44a9-9b23-510f7279d3c9 service neutron] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:20:48.835358 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1238/2467] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:20:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:20:51.927352 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07638b5-c4f0-4dfa-a4be-f949a534c814 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:51.928575 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07638b5-c4f0-4dfa-a4be-f949a534c814 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:51.929009 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07638b5-c4f0-4dfa-a4be-f949a534c814 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:51.931481 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1230/2468] 10.222.0.61 () {54 vars in 961 bytes} [Fri Aug 26 12:20:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:20:51.937397 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64e9cd3-bce6-4f83-92b3-6cdf295e1071 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:51.937872 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64e9cd3-bce6-4f83-92b3-6cdf295e1071 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:51.938090 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64e9cd3-bce6-4f83-92b3-6cdf295e1071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:51.960351 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-a64e9cd3-bce6-4f83-92b3-6cdf295e1071 None None] MFA Rules not processed for user `7346e4a29dfb4a0a9768597704be286c`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:20:51.961581 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-a64e9cd3-bce6-4f83-92b3-6cdf295e1071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:20:51.989476 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07638b5-c4f0-4dfa-a4be-f949a534c814 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:20:51.991318 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-a07638b5-c4f0-4dfa-a4be-f949a534c814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:20:52.020384 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07638b5-c4f0-4dfa-a4be-f949a534c814 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:20:52.028939 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ffddc3-d29b-47cd-b006-b0767ba88378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:20:52.030721 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ffddc3-d29b-47cd-b006-b0767ba88378 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:52.030852 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ffddc3-d29b-47cd-b006-b0767ba88378 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:52.031174 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ffddc3-d29b-47cd-b006-b0767ba88378 admin admin] PATH_INFO: `/v3/users/2d9f1a5e4e77467f8a691b634a841de2` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:52.033030 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ffddc3-d29b-47cd-b006-b0767ba88378 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d9f1a5e4e77467f8a691b634a841de2)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:20:52.035076 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ffddc3-d29b-47cd-b006-b0767ba88378 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:20:52.086177 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1239/2469] 10.222.0.61 () {58 vars in 1058 bytes} [Fri Aug 26 12:20:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:20:52.094758 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64e9cd3-bce6-4f83-92b3-6cdf295e1071 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:20:52.100464 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-d3ffddc3-d29b-47cd-b006-b0767ba88378 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d9f1a5e4e77467f8a691b634a841de2'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:52.108211 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-d3ffddc3-d29b-47cd-b006-b0767ba88378 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d9f1a5e4e77467f8a691b634a841de2'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:52.109539 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64e9cd3-bce6-4f83-92b3-6cdf295e1071 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:20:52.124172 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced37ff-3538-4b39-8863-625be8d15554 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7346e4a29dfb4a0a9768597704be286c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:20:52.124727 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-d3ffddc3-d29b-47cd-b006-b0767ba88378 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d9f1a5e4e77467f8a691b634a841de2'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:52.124862 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced37ff-3538-4b39-8863-625be8d15554 service placement] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:52.125057 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced37ff-3538-4b39-8863-625be8d15554 service placement] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:52.125324 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced37ff-3538-4b39-8863-625be8d15554 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:52.125494 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-d3ffddc3-d29b-47cd-b006-b0767ba88378 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d9f1a5e4e77467f8a691b634a841de2'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:52.127070 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-8ced37ff-3538-4b39-8863-625be8d15554 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:20:52.128029 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=79,keystone:INSERT=3,keystone:DELETE=18 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:20:52.143332 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ced37ff-3538-4b39-8863-625be8d15554 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:20:52.143672 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-d3ffddc3-d29b-47cd-b006-b0767ba88378 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d9f1a5e4e77467f8a691b634a841de2'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:52.145048 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ced37ff-3538-4b39-8863-625be8d15554 service placement] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:20:52.150932 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1231/2470] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:20:51 2022] DELETE /identity/v3/users/2d9f1a5e4e77467f8a691b634a841de2 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:20:52.156980 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1240/2471] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:20:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 12:20:52.351595 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ffddc3-d29b-47cd-b006-b0767ba88378 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:20:52.353168 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-d3ffddc3-d29b-47cd-b006-b0767ba88378 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:20:52.444402 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ffddc3-d29b-47cd-b006-b0767ba88378 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:20:52.449169 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94e58d7-c10b-4d0b-ac8b-b35c8077adba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:20:52.449879 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94e58d7-c10b-4d0b-ac8b-b35c8077adba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:52.450385 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94e58d7-c10b-4d0b-ac8b-b35c8077adba admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:52.450617 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94e58d7-c10b-4d0b-ac8b-b35c8077adba admin admin] PATH_INFO: `/v3/projects/99e42555cbcd4716a1be1075f4a2b3b1` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:52.451916 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94e58d7-c10b-4d0b-ac8b-b35c8077adba admin admin] RBAC: Authorizing `identity:delete_project(project_id=99e42555cbcd4716a1be1075f4a2b3b1)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:20:52.454048 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94e58d7-c10b-4d0b-ac8b-b35c8077adba admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:20:52.500573 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-b94e58d7-c10b-4d0b-ac8b-b35c8077adba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99e42555cbcd4716a1be1075f4a2b3b1', 'request_id': 'req-b94e58d7-c10b-4d0b-ac8b-b35c8077adba'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:52.506551 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-b94e58d7-c10b-4d0b-ac8b-b35c8077adba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99e42555cbcd4716a1be1075f4a2b3b1', 'request_id': 'req-b94e58d7-c10b-4d0b-ac8b-b35c8077adba'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:52.513780 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-b94e58d7-c10b-4d0b-ac8b-b35c8077adba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99e42555cbcd4716a1be1075f4a2b3b1', 'request_id': 'req-b94e58d7-c10b-4d0b-ac8b-b35c8077adba'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:52.524159 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-b94e58d7-c10b-4d0b-ac8b-b35c8077adba admin admin] The token cache is being invalidate because project 99e42555cbcd4716a1be1075f4a2b3b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:20:52.524451 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-b94e58d7-c10b-4d0b-ac8b-b35c8077adba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:52.525071 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-b94e58d7-c10b-4d0b-ac8b-b35c8077adba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:52.526541 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1232/2472] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:20:52 2022] DELETE /identity/v3/projects/99e42555cbcd4716a1be1075f4a2b3b1 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:20:52.531456 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab732d7d-5fb3-415b-8f62-3c1c9a7f2c72 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:52.531848 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab732d7d-5fb3-415b-8f62-3c1c9a7f2c72 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:52.532193 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab732d7d-5fb3-415b-8f62-3c1c9a7f2c72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:52.551971 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-ab732d7d-5fb3-415b-8f62-3c1c9a7f2c72 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:20:52.553165 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-ab732d7d-5fb3-415b-8f62-3c1c9a7f2c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:20:52.612143 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1241/2473] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:20:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:20:52.620256 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1652e36-bde7-45ba-a5f8-c0def2771db1 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:52.620641 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1652e36-bde7-45ba-a5f8-c0def2771db1 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:52.620984 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1652e36-bde7-45ba-a5f8-c0def2771db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:52.641419 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-a1652e36-bde7-45ba-a5f8-c0def2771db1 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:20:52.642608 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-a1652e36-bde7-45ba-a5f8-c0def2771db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:20:52.650407 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1233/2474] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:20:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:20:52.654526 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab732d7d-5fb3-415b-8f62-3c1c9a7f2c72 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:20:52.664644 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab732d7d-5fb3-415b-8f62-3c1c9a7f2c72 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:20:52.669221 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744c5a46-6948-466b-874c-e9e493afa4db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:20:52.669806 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744c5a46-6948-466b-874c-e9e493afa4db admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:52.670086 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744c5a46-6948-466b-874c-e9e493afa4db admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:52.670357 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744c5a46-6948-466b-874c-e9e493afa4db admin admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:52.671567 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744c5a46-6948-466b-874c-e9e493afa4db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:20:52.671871 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744c5a46-6948-466b-874c-e9e493afa4db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:20:52.673446 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744c5a46-6948-466b-874c-e9e493afa4db admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:20:52.681918 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:20:52.685644 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1242/2475] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:20:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:20:52.690111 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1652e36-bde7-45ba-a5f8-c0def2771db1 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:20:52.699142 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1652e36-bde7-45ba-a5f8-c0def2771db1 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:20:52.704759 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3001d6af-72c0-4186-8e67-e99b1afeca21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:20:52.705348 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3001d6af-72c0-4186-8e67-e99b1afeca21 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:52.705625 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3001d6af-72c0-4186-8e67-e99b1afeca21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:52.705893 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3001d6af-72c0-4186-8e67-e99b1afeca21 admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:52.706514 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3001d6af-72c0-4186-8e67-e99b1afeca21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:20:52.708416 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3001d6af-72c0-4186-8e67-e99b1afeca21 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:20:52.734040 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1234/2476] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:20:52 2022] POST /identity/v3/projects => generated 371 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:20:52.738173 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744c5a46-6948-466b-874c-e9e493afa4db admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:20:52.751929 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744c5a46-6948-466b-874c-e9e493afa4db admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:20:52.807657 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e53772-1bdf-4892-bcfb-bb7ebe2b0edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:20:52.808255 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e53772-1bdf-4892-bcfb-bb7ebe2b0edc admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:52.808537 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e53772-1bdf-4892-bcfb-bb7ebe2b0edc admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:52.808839 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e53772-1bdf-4892-bcfb-bb7ebe2b0edc admin admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:52.809467 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e53772-1bdf-4892-bcfb-bb7ebe2b0edc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:20:52.811939 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e53772-1bdf-4892-bcfb-bb7ebe2b0edc admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:20:52.833624 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1243/2477] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:20:52 2022] POST /identity/v3/users => generated 286 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:20:52.837825 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3001d6af-72c0-4186-8e67-e99b1afeca21 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:20:52.848134 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3001d6af-72c0-4186-8e67-e99b1afeca21 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:20:52.852659 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d88e6cf-5f80-40bf-99e7-ffbd4aaacb2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:20:52.853248 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d88e6cf-5f80-40bf-99e7-ffbd4aaacb2b admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:52.853525 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d88e6cf-5f80-40bf-99e7-ffbd4aaacb2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:52.853791 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d88e6cf-5f80-40bf-99e7-ffbd4aaacb2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:52.854423 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d88e6cf-5f80-40bf-99e7-ffbd4aaacb2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:20:52.856032 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d88e6cf-5f80-40bf-99e7-ffbd4aaacb2b admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:20:52.866449 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1235/2478] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:20:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:20:52.870302 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e53772-1bdf-4892-bcfb-bb7ebe2b0edc admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:20:52.881191 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e53772-1bdf-4892-bcfb-bb7ebe2b0edc admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:20:52.885848 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538e8535-035e-4846-b537-85fab3d432cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:20:52.886458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e8535-035e-4846-b537-85fab3d432cc admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:52.886711 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e8535-035e-4846-b537-85fab3d432cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:52.886983 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e8535-035e-4846-b537-85fab3d432cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:52.887647 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538e8535-035e-4846-b537-85fab3d432cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:20:52.889829 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538e8535-035e-4846-b537-85fab3d432cc admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:20:52.900310 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1244/2479] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:20:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 12:20:52.904458 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d88e6cf-5f80-40bf-99e7-ffbd4aaacb2b admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:20:52.914657 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d88e6cf-5f80-40bf-99e7-ffbd4aaacb2b admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:20:52.919263 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34806e1b-e03e-4423-b1b4-afd5db58f4cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:20:52.919899 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34806e1b-e03e-4423-b1b4-afd5db58f4cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:52.920178 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34806e1b-e03e-4423-b1b4-afd5db58f4cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:52.920494 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34806e1b-e03e-4423-b1b4-afd5db58f4cb admin admin] PATH_INFO: `/v3/projects/4cadc98f208d41dca49fab94acd9cee8/users/c8322cd70f7343f99db9765adbbcb7f6/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:52.934567 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34806e1b-e03e-4423-b1b4-afd5db58f4cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cadc98f208d41dca49fab94acd9cee8, user_id=c8322cd70f7343f99db9765adbbcb7f6, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:20:52.937842 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34806e1b-e03e-4423-b1b4-afd5db58f4cb admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:20:52.950257 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1236/2480] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:20:52 2022] PUT /identity/v3/projects/4cadc98f208d41dca49fab94acd9cee8/users/c8322cd70f7343f99db9765adbbcb7f6/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:20:52.962522 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d505f2-4b23-40e5-adf8-e7a6ad71304d None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:52.962987 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d505f2-4b23-40e5-adf8-e7a6ad71304d None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:52.963128 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d505f2-4b23-40e5-adf8-e7a6ad71304d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:52.992790 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-d5d505f2-4b23-40e5-adf8-e7a6ad71304d None None] MFA Rules not processed for user `c8322cd70f7343f99db9765adbbcb7f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:20:52.994034 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-d5d505f2-4b23-40e5-adf8-e7a6ad71304d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:20:53.056882 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1245/2481] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:20:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:20:53.062093 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34806e1b-e03e-4423-b1b4-afd5db58f4cb admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:20:53.072119 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:20:53.125491 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34806e1b-e03e-4423-b1b4-afd5db58f4cb admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:20:53.130145 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4279505b-c713-48a6-ab69-8eb6a2c8a23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:20:53.130799 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279505b-c713-48a6-ab69-8eb6a2c8a23a admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:53.131086 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279505b-c713-48a6-ab69-8eb6a2c8a23a admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:53.131393 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279505b-c713-48a6-ab69-8eb6a2c8a23a admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:53.132010 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4279505b-c713-48a6-ab69-8eb6a2c8a23a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:20:53.133905 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4279505b-c713-48a6-ab69-8eb6a2c8a23a admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:20:53.159351 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1237/2482] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:20:53 2022] POST /identity/v3/projects => generated 373 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:20:53.163411 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d505f2-4b23-40e5-adf8-e7a6ad71304d None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:20:53.175068 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d505f2-4b23-40e5-adf8-e7a6ad71304d None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:20:53.230445 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903d5b05-f5b3-4ec8-b9a0-31bf6a3c9513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:20:53.230974 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903d5b05-f5b3-4ec8-b9a0-31bf6a3c9513 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:53.231278 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903d5b05-f5b3-4ec8-b9a0-31bf6a3c9513 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:53.231559 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903d5b05-f5b3-4ec8-b9a0-31bf6a3c9513 admin admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:53.232190 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903d5b05-f5b3-4ec8-b9a0-31bf6a3c9513 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:20:53.234589 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903d5b05-f5b3-4ec8-b9a0-31bf6a3c9513 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:20:53.250699 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:20:53.259930 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1246/2483] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:20:53 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:20:53.261110 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4279505b-c713-48a6-ab69-8eb6a2c8a23a admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:20:53.272002 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4279505b-c713-48a6-ab69-8eb6a2c8a23a admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:20:53.276645 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d93d1d7-bad5-4242-9f74-302853796a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:20:53.277223 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93d1d7-bad5-4242-9f74-302853796a99 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:53.277502 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93d1d7-bad5-4242-9f74-302853796a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:53.277773 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93d1d7-bad5-4242-9f74-302853796a99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:53.278401 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d93d1d7-bad5-4242-9f74-302853796a99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:20:53.280399 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d93d1d7-bad5-4242-9f74-302853796a99 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:20:53.291500 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1238/2484] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:20:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:20:53.296257 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903d5b05-f5b3-4ec8-b9a0-31bf6a3c9513 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:20:53.306124 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903d5b05-f5b3-4ec8-b9a0-31bf6a3c9513 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:20:53.310725 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c40df4-74b7-4bb2-a792-a648fdcbc213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:20:53.312106 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c40df4-74b7-4bb2-a792-a648fdcbc213 admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:53.312395 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c40df4-74b7-4bb2-a792-a648fdcbc213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:53.312667 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c40df4-74b7-4bb2-a792-a648fdcbc213 admin admin] PATH_INFO: `/v3/projects/c6d04e225498459c895030616fe907fd/users/b9d0b1f1b31b4b90a44fa486b99493d1/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:53.327021 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c40df4-74b7-4bb2-a792-a648fdcbc213 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6d04e225498459c895030616fe907fd, user_id=b9d0b1f1b31b4b90a44fa486b99493d1, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:20:53.329699 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c40df4-74b7-4bb2-a792-a648fdcbc213 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:20:53.341951 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1247/2485] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:20:53 2022] PUT /identity/v3/projects/c6d04e225498459c895030616fe907fd/users/b9d0b1f1b31b4b90a44fa486b99493d1/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:20:53.346717 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d93d1d7-bad5-4242-9f74-302853796a99 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:20:53.356657 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d93d1d7-bad5-4242-9f74-302853796a99 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:20:53.413498 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbf7231-812f-4455-afa9-f08728cee084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:20:53.414099 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf7231-812f-4455-afa9-f08728cee084 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:53.414376 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf7231-812f-4455-afa9-f08728cee084 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:53.414645 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf7231-812f-4455-afa9-f08728cee084 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:53.415295 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbf7231-812f-4455-afa9-f08728cee084 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:20:53.416877 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbf7231-812f-4455-afa9-f08728cee084 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:20:53.426898 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1239/2486] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:20:53 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:20:53.431033 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c40df4-74b7-4bb2-a792-a648fdcbc213 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:20:53.441284 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c40df4-74b7-4bb2-a792-a648fdcbc213 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:20:53.445763 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d88c1c-b97f-483d-a0c3-4b6d68161497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:20:53.446394 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d88c1c-b97f-483d-a0c3-4b6d68161497 admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:53.446670 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d88c1c-b97f-483d-a0c3-4b6d68161497 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:53.446940 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d88c1c-b97f-483d-a0c3-4b6d68161497 admin admin] PATH_INFO: `/v3/projects/c6d04e225498459c895030616fe907fd/users/b9d0b1f1b31b4b90a44fa486b99493d1/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:53.448853 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d88c1c-b97f-483d-a0c3-4b6d68161497 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6d04e225498459c895030616fe907fd, user_id=b9d0b1f1b31b4b90a44fa486b99493d1, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:20:53.451489 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d88c1c-b97f-483d-a0c3-4b6d68161497 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:20:53.461731 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1248/2487] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:20:53 2022] PUT /identity/v3/projects/c6d04e225498459c895030616fe907fd/users/b9d0b1f1b31b4b90a44fa486b99493d1/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:20:53.472461 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e597f2-c4a6-4987-bdde-a6318eb9a194 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:53.472755 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e597f2-c4a6-4987-bdde-a6318eb9a194 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:53.473063 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e597f2-c4a6-4987-bdde-a6318eb9a194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:53.502437 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-f0e597f2-c4a6-4987-bdde-a6318eb9a194 None None] MFA Rules not processed for user `b9d0b1f1b31b4b90a44fa486b99493d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:20:53.503677 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-f0e597f2-c4a6-4987-bdde-a6318eb9a194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:20:53.574576 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1240/2488] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:20:53 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:20:53.586006 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d88c1c-b97f-483d-a0c3-4b6d68161497 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:20:53.587860 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-94d88c1c-b97f-483d-a0c3-4b6d68161497 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:20:53.647541 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d88c1c-b97f-483d-a0c3-4b6d68161497 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:20:53.651103 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca8b91d-47a4-44f8-8f34-3024b646ba55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:20:53.651724 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca8b91d-47a4-44f8-8f34-3024b646ba55 service cinder] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:53.651999 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca8b91d-47a4-44f8-8f34-3024b646ba55 service cinder] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:53.652266 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca8b91d-47a4-44f8-8f34-3024b646ba55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:53.661044 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca8b91d-47a4-44f8-8f34-3024b646ba55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:20:53.662444 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca8b91d-47a4-44f8-8f34-3024b646ba55 service cinder] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:20:53.671934 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1249/2489] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:20:53 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:20:54.060770 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e597f2-c4a6-4987-bdde-a6318eb9a194 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:20:54.066868 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:20:54.125145 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e597f2-c4a6-4987-bdde-a6318eb9a194 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:20:54.129677 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fab41a-8f01-4be8-866f-3d1d9c42181c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:20:54.130300 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fab41a-8f01-4be8-866f-3d1d9c42181c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:54.130579 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fab41a-8f01-4be8-866f-3d1d9c42181c admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:54.130850 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fab41a-8f01-4be8-866f-3d1d9c42181c admin admin] PATH_INFO: `/v3/users/c8322cd70f7343f99db9765adbbcb7f6` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:54.131995 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fab41a-8f01-4be8-866f-3d1d9c42181c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8322cd70f7343f99db9765adbbcb7f6)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:20:54.133894 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fab41a-8f01-4be8-866f-3d1d9c42181c admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:20:54.178644 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-96fab41a-8f01-4be8-866f-3d1d9c42181c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8322cd70f7343f99db9765adbbcb7f6'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:54.185610 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-96fab41a-8f01-4be8-866f-3d1d9c42181c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8322cd70f7343f99db9765adbbcb7f6'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:54.192267 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-96fab41a-8f01-4be8-866f-3d1d9c42181c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8322cd70f7343f99db9765adbbcb7f6'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:54.192890 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-96fab41a-8f01-4be8-866f-3d1d9c42181c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8322cd70f7343f99db9765adbbcb7f6'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:54.204295 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-96fab41a-8f01-4be8-866f-3d1d9c42181c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8322cd70f7343f99db9765adbbcb7f6'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:54.209938 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1241/2490] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:20:54 2022] DELETE /identity/v3/users/c8322cd70f7343f99db9765adbbcb7f6 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:20:54.214332 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675c637c-efbb-41a9-b0e3-d6b160e27fd8 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:54.214846 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675c637c-efbb-41a9-b0e3-d6b160e27fd8 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:54.214846 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675c637c-efbb-41a9-b0e3-d6b160e27fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:54.234321 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-675c637c-efbb-41a9-b0e3-d6b160e27fd8 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:20:54.235536 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-675c637c-efbb-41a9-b0e3-d6b160e27fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:20:54.294385 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1250/2491] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:20:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:20:54.304847 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fab41a-8f01-4be8-866f-3d1d9c42181c admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:20:54.306216 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-96fab41a-8f01-4be8-866f-3d1d9c42181c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:20:54.362759 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fab41a-8f01-4be8-866f-3d1d9c42181c admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:20:54.366620 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed70f54-2112-4d5c-bf06-c82b8120c890 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:20:54.367244 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed70f54-2112-4d5c-bf06-c82b8120c890 service neutron] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:54.367527 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed70f54-2112-4d5c-bf06-c82b8120c890 service neutron] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:54.367799 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed70f54-2112-4d5c-bf06-c82b8120c890 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:54.376792 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed70f54-2112-4d5c-bf06-c82b8120c890 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:20:54.378218 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed70f54-2112-4d5c-bf06-c82b8120c890 service neutron] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:20:54.387468 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1242/2492] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:20:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:20:54.649535 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675c637c-efbb-41a9-b0e3-d6b160e27fd8 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:20:54.651375 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-675c637c-efbb-41a9-b0e3-d6b160e27fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:20:54.664798 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675c637c-efbb-41a9-b0e3-d6b160e27fd8 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:20:54.669946 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aa9f68-809d-453b-9695-bc8a08912f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:20:54.670744 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aa9f68-809d-453b-9695-bc8a08912f23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:54.671030 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aa9f68-809d-453b-9695-bc8a08912f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:54.671335 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aa9f68-809d-453b-9695-bc8a08912f23 admin admin] PATH_INFO: `/v3/projects/4cadc98f208d41dca49fab94acd9cee8` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:54.672484 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aa9f68-809d-453b-9695-bc8a08912f23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cadc98f208d41dca49fab94acd9cee8)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:20:54.674457 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aa9f68-809d-453b-9695-bc8a08912f23 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:20:54.727285 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-02aa9f68-809d-453b-9695-bc8a08912f23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cadc98f208d41dca49fab94acd9cee8', 'request_id': 'req-02aa9f68-809d-453b-9695-bc8a08912f23'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:54.733597 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-02aa9f68-809d-453b-9695-bc8a08912f23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cadc98f208d41dca49fab94acd9cee8', 'request_id': 'req-02aa9f68-809d-453b-9695-bc8a08912f23'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:54.737431 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:20:54.749349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-02aa9f68-809d-453b-9695-bc8a08912f23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cadc98f208d41dca49fab94acd9cee8', 'request_id': 'req-02aa9f68-809d-453b-9695-bc8a08912f23'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:54.765225 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-02aa9f68-809d-453b-9695-bc8a08912f23 admin admin] The token cache is being invalidate because project 4cadc98f208d41dca49fab94acd9cee8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81712) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:20:54.765596 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-02aa9f68-809d-453b-9695-bc8a08912f23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:54.766359 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-02aa9f68-809d-453b-9695-bc8a08912f23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:54.768024 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1251/2493] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:20:54 2022] DELETE /identity/v3/projects/4cadc98f208d41dca49fab94acd9cee8 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:20:54.772211 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed70f54-2112-4d5c-bf06-c82b8120c890 service neutron] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:20:54.782998 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed70f54-2112-4d5c-bf06-c82b8120c890 service neutron] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:20:54.808152 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:20:54.848194 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ea7d18-ce71-420a-8cfa-8e58d07591be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:20:54.848742 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ea7d18-ce71-420a-8cfa-8e58d07591be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:54.849018 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ea7d18-ce71-420a-8cfa-8e58d07591be admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:54.849288 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ea7d18-ce71-420a-8cfa-8e58d07591be admin admin] PATH_INFO: `/v3/users/b9d0b1f1b31b4b90a44fa486b99493d1` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:54.850385 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ea7d18-ce71-420a-8cfa-8e58d07591be admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9d0b1f1b31b4b90a44fa486b99493d1)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:20:54.852324 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ea7d18-ce71-420a-8cfa-8e58d07591be admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:20:54.899049 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-48ea7d18-ce71-420a-8cfa-8e58d07591be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9d0b1f1b31b4b90a44fa486b99493d1'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:54.906459 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-48ea7d18-ce71-420a-8cfa-8e58d07591be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9d0b1f1b31b4b90a44fa486b99493d1'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:54.913791 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-48ea7d18-ce71-420a-8cfa-8e58d07591be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9d0b1f1b31b4b90a44fa486b99493d1'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:54.914421 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-48ea7d18-ce71-420a-8cfa-8e58d07591be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9d0b1f1b31b4b90a44fa486b99493d1'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:54.926429 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-48ea7d18-ce71-420a-8cfa-8e58d07591be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9d0b1f1b31b4b90a44fa486b99493d1'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:54.932213 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1243/2494] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:20:54 2022] DELETE /identity/v3/users/b9d0b1f1b31b4b90a44fa486b99493d1 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:20:55.200670 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aa9f68-809d-453b-9695-bc8a08912f23 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:20:55.203035 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-02aa9f68-809d-453b-9695-bc8a08912f23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:20:55.217306 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aa9f68-809d-453b-9695-bc8a08912f23 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:20:55.282343 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4df7fb-9252-4347-8d03-341de89c9f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:20:55.282968 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4df7fb-9252-4347-8d03-341de89c9f02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:55.283272 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4df7fb-9252-4347-8d03-341de89c9f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:55.283547 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4df7fb-9252-4347-8d03-341de89c9f02 admin admin] PATH_INFO: `/v3/projects/c6d04e225498459c895030616fe907fd` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:55.284654 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4df7fb-9252-4347-8d03-341de89c9f02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6d04e225498459c895030616fe907fd)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:20:55.286519 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4df7fb-9252-4347-8d03-341de89c9f02 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:20:55.332018 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-cd4df7fb-9252-4347-8d03-341de89c9f02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6d04e225498459c895030616fe907fd', 'request_id': 'req-cd4df7fb-9252-4347-8d03-341de89c9f02'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:55.338058 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-cd4df7fb-9252-4347-8d03-341de89c9f02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6d04e225498459c895030616fe907fd', 'request_id': 'req-cd4df7fb-9252-4347-8d03-341de89c9f02'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:55.345618 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-cd4df7fb-9252-4347-8d03-341de89c9f02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6d04e225498459c895030616fe907fd', 'request_id': 'req-cd4df7fb-9252-4347-8d03-341de89c9f02'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:55.355921 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-cd4df7fb-9252-4347-8d03-341de89c9f02 admin admin] The token cache is being invalidate because project c6d04e225498459c895030616fe907fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81712) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:20:55.356209 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-cd4df7fb-9252-4347-8d03-341de89c9f02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:55.356829 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-cd4df7fb-9252-4347-8d03-341de89c9f02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:20:55.358826 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1252/2495] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:20:55 2022] DELETE /identity/v3/projects/c6d04e225498459c895030616fe907fd => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:20:55.363995 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df78f1e-a422-4c52-a2a5-142a5f81c081 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:55.364443 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df78f1e-a422-4c52-a2a5-142a5f81c081 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:55.364535 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df78f1e-a422-4c52-a2a5-142a5f81c081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:55.384295 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-4df78f1e-a422-4c52-a2a5-142a5f81c081 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:20:55.385417 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-4df78f1e-a422-4c52-a2a5-142a5f81c081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:20:55.442945 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1244/2496] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:20:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:20:55.450461 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a7137-8237-4962-abd9-002e2a462ba9 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:55.450755 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a7137-8237-4962-abd9-002e2a462ba9 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:55.451039 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a7137-8237-4962-abd9-002e2a462ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:55.470463 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-1c8a7137-8237-4962-abd9-002e2a462ba9 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:20:55.471689 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-1c8a7137-8237-4962-abd9-002e2a462ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:20:55.479763 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1253/2497] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:20:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:20:55.484385 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df78f1e-a422-4c52-a2a5-142a5f81c081 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:20:55.494383 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df78f1e-a422-4c52-a2a5-142a5f81c081 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:20:55.499630 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664b4ab1-23dc-40df-9ce7-99b1d8f03f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:20:55.500277 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b4ab1-23dc-40df-9ce7-99b1d8f03f19 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:55.500562 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b4ab1-23dc-40df-9ce7-99b1d8f03f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:55.500837 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b4ab1-23dc-40df-9ce7-99b1d8f03f19 admin admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:55.501546 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664b4ab1-23dc-40df-9ce7-99b1d8f03f19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:20:55.501856 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664b4ab1-23dc-40df-9ce7-99b1d8f03f19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:20:55.503484 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664b4ab1-23dc-40df-9ce7-99b1d8f03f19 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:20:55.515159 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1245/2498] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:20:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:20:55.519486 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a7137-8237-4962-abd9-002e2a462ba9 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:20:55.528550 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a7137-8237-4962-abd9-002e2a462ba9 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:20:55.533240 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fc812f-8861-4f0f-9c30-039d6fe2337f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:20:55.533819 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc812f-8861-4f0f-9c30-039d6fe2337f admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:55.534095 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc812f-8861-4f0f-9c30-039d6fe2337f admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:55.534415 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc812f-8861-4f0f-9c30-039d6fe2337f admin admin] PATH_INFO: `/v3/projects` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:55.535020 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fc812f-8861-4f0f-9c30-039d6fe2337f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:20:55.536928 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fc812f-8861-4f0f-9c30-039d6fe2337f admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:20:55.565889 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1254/2499] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:20:55 2022] POST /identity/v3/projects => generated 357 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:20:55.570319 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664b4ab1-23dc-40df-9ce7-99b1d8f03f19 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:20:55.580689 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664b4ab1-23dc-40df-9ce7-99b1d8f03f19 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:20:55.610847 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:20:55.638386 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2946648e-bee2-47a1-bd53-be93e50f85d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:20:55.638916 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2946648e-bee2-47a1-bd53-be93e50f85d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:55.639250 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2946648e-bee2-47a1-bd53-be93e50f85d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:55.639531 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2946648e-bee2-47a1-bd53-be93e50f85d4 admin admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:55.640148 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2946648e-bee2-47a1-bd53-be93e50f85d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:20:55.642056 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2946648e-bee2-47a1-bd53-be93e50f85d4 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:20:55.663127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1246/2500] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:20:55 2022] POST /identity/v3/users => generated 279 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:20:55.667272 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fc812f-8861-4f0f-9c30-039d6fe2337f admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:20:55.678995 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fc812f-8861-4f0f-9c30-039d6fe2337f admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:20:55.683620 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50acd643-1a16-4b2a-b5ca-18dbab5f0edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:20:55.684208 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50acd643-1a16-4b2a-b5ca-18dbab5f0edb admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:55.684485 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50acd643-1a16-4b2a-b5ca-18dbab5f0edb admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:55.684755 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50acd643-1a16-4b2a-b5ca-18dbab5f0edb admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:55.685384 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50acd643-1a16-4b2a-b5ca-18dbab5f0edb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:20:55.686988 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50acd643-1a16-4b2a-b5ca-18dbab5f0edb admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:20:55.696796 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1255/2501] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:20:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:20:55.700616 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2946648e-bee2-47a1-bd53-be93e50f85d4 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:20:55.710470 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2946648e-bee2-47a1-bd53-be93e50f85d4 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:20:55.715006 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eff005a-0da4-4333-b92d-e74bd0e662c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:20:55.715639 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff005a-0da4-4333-b92d-e74bd0e662c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:55.715922 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff005a-0da4-4333-b92d-e74bd0e662c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:55.716196 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff005a-0da4-4333-b92d-e74bd0e662c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:55.716830 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eff005a-0da4-4333-b92d-e74bd0e662c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:20:55.718931 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eff005a-0da4-4333-b92d-e74bd0e662c1 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:20:55.729236 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1247/2502] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:20:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:20:55.733880 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50acd643-1a16-4b2a-b5ca-18dbab5f0edb admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:20:55.744140 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50acd643-1a16-4b2a-b5ca-18dbab5f0edb admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:20:55.748844 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28950e77-dcf1-44eb-8d3b-cf1aa396396f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:20:55.749714 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28950e77-dcf1-44eb-8d3b-cf1aa396396f admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:55.749995 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28950e77-dcf1-44eb-8d3b-cf1aa396396f admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:55.750271 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28950e77-dcf1-44eb-8d3b-cf1aa396396f admin admin] PATH_INFO: `/v3/projects/ab431fd0cf9549d6b24bc49984628214/users/98c7d230233c4743aa8fccd885f1f04e/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:55.765070 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28950e77-dcf1-44eb-8d3b-cf1aa396396f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab431fd0cf9549d6b24bc49984628214, user_id=98c7d230233c4743aa8fccd885f1f04e, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:20:55.767747 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28950e77-dcf1-44eb-8d3b-cf1aa396396f admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:20:55.781367 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1256/2503] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:20:55 2022] PUT /identity/v3/projects/ab431fd0cf9549d6b24bc49984628214/users/98c7d230233c4743aa8fccd885f1f04e/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:20:55.792626 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d0a74-2c54-49bc-ad18-e4b7b88f4a62 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:55.792912 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d0a74-2c54-49bc-ad18-e4b7b88f4a62 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:55.793211 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d0a74-2c54-49bc-ad18-e4b7b88f4a62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:55.820666 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-dd4d0a74-2c54-49bc-ad18-e4b7b88f4a62 None None] MFA Rules not processed for user `98c7d230233c4743aa8fccd885f1f04e`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:20:55.821868 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-dd4d0a74-2c54-49bc-ad18-e4b7b88f4a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:20:55.886393 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1248/2504] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:20:55 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:20:55.891509 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28950e77-dcf1-44eb-8d3b-cf1aa396396f admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:20:55.908710 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:20:55.960190 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28950e77-dcf1-44eb-8d3b-cf1aa396396f admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:20:55.965013 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1091d39-6d09-41b1-be31-a10860e1bead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:20:55.965609 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1091d39-6d09-41b1-be31-a10860e1bead admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:55.965885 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1091d39-6d09-41b1-be31-a10860e1bead admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:55.966155 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1091d39-6d09-41b1-be31-a10860e1bead admin admin] PATH_INFO: `/v3/projects` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:55.966780 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1091d39-6d09-41b1-be31-a10860e1bead admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:20:55.969569 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1091d39-6d09-41b1-be31-a10860e1bead admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:20:56.000100 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1257/2505] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:20:55 2022] POST /identity/v3/projects => generated 355 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:20:56.004250 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4d0a74-2c54-49bc-ad18-e4b7b88f4a62 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:20:56.015663 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4d0a74-2c54-49bc-ad18-e4b7b88f4a62 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:20:56.070655 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2168eed6-ecf4-46d9-afd8-75f81b83b718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:20:56.071284 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168eed6-ecf4-46d9-afd8-75f81b83b718 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:56.071572 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168eed6-ecf4-46d9-afd8-75f81b83b718 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:56.071847 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168eed6-ecf4-46d9-afd8-75f81b83b718 admin admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:56.072477 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2168eed6-ecf4-46d9-afd8-75f81b83b718 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:20:56.074388 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2168eed6-ecf4-46d9-afd8-75f81b83b718 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:20:56.095037 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1249/2506] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:20:56 2022] POST /identity/v3/users => generated 284 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:20:56.099134 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1091d39-6d09-41b1-be31-a10860e1bead admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:20:56.109906 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1091d39-6d09-41b1-be31-a10860e1bead admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:20:56.114929 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8d1043-59e6-4538-b824-cbec2311a5aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:20:56.115574 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d1043-59e6-4538-b824-cbec2311a5aa admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:56.115838 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d1043-59e6-4538-b824-cbec2311a5aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:56.116112 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d1043-59e6-4538-b824-cbec2311a5aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:56.116747 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8d1043-59e6-4538-b824-cbec2311a5aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:20:56.118334 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8d1043-59e6-4538-b824-cbec2311a5aa admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:20:56.129774 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1258/2507] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:20:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:20:56.133814 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2168eed6-ecf4-46d9-afd8-75f81b83b718 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:20:56.143894 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2168eed6-ecf4-46d9-afd8-75f81b83b718 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:20:56.148542 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95112b4-22b1-42d7-8da1-cae63b374257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:20:56.149215 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95112b4-22b1-42d7-8da1-cae63b374257 admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:56.149515 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95112b4-22b1-42d7-8da1-cae63b374257 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:56.149783 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95112b4-22b1-42d7-8da1-cae63b374257 admin admin] PATH_INFO: `/v3/projects/3905fc2662ad402fb9621844541a6f06/users/ec26eeb74e7b48858ca08434a1329da1/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:56.164234 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95112b4-22b1-42d7-8da1-cae63b374257 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3905fc2662ad402fb9621844541a6f06, user_id=ec26eeb74e7b48858ca08434a1329da1, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:20:56.166886 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95112b4-22b1-42d7-8da1-cae63b374257 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:20:56.179895 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1250/2508] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:20:56 2022] PUT /identity/v3/projects/3905fc2662ad402fb9621844541a6f06/users/ec26eeb74e7b48858ca08434a1329da1/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:20:56.183939 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8d1043-59e6-4538-b824-cbec2311a5aa admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:20:56.194389 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8d1043-59e6-4538-b824-cbec2311a5aa admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:20:56.250072 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f94e580-4da7-4870-a982-7f3742b1e092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:20:56.250703 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94e580-4da7-4870-a982-7f3742b1e092 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:56.250932 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94e580-4da7-4870-a982-7f3742b1e092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:56.251224 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94e580-4da7-4870-a982-7f3742b1e092 admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:56.251864 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f94e580-4da7-4870-a982-7f3742b1e092 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:20:56.253459 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f94e580-4da7-4870-a982-7f3742b1e092 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:20:56.263125 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1259/2509] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:20:56 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:20:56.267607 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95112b4-22b1-42d7-8da1-cae63b374257 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:20:56.277800 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95112b4-22b1-42d7-8da1-cae63b374257 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:20:56.282485 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b76d9b-d02c-4407-a68e-3292e9d6b59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:20:56.283131 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b76d9b-d02c-4407-a68e-3292e9d6b59d admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:56.283437 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b76d9b-d02c-4407-a68e-3292e9d6b59d admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:56.283708 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b76d9b-d02c-4407-a68e-3292e9d6b59d admin admin] PATH_INFO: `/v3/projects/3905fc2662ad402fb9621844541a6f06/users/ec26eeb74e7b48858ca08434a1329da1/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:56.285647 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b76d9b-d02c-4407-a68e-3292e9d6b59d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3905fc2662ad402fb9621844541a6f06, user_id=ec26eeb74e7b48858ca08434a1329da1, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:20:56.288296 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b76d9b-d02c-4407-a68e-3292e9d6b59d admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:20:56.290436 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:20:56.300527 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1251/2510] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:20:56 2022] PUT /identity/v3/projects/3905fc2662ad402fb9621844541a6f06/users/ec26eeb74e7b48858ca08434a1329da1/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 12:20:56.310472 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ad4f1-1637-4b50-81ce-312697058ff2 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:56.310759 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ad4f1-1637-4b50-81ce-312697058ff2 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:56.311682 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ad4f1-1637-4b50-81ce-312697058ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:56.338508 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-638ad4f1-1637-4b50-81ce-312697058ff2 None None] MFA Rules not processed for user `ec26eeb74e7b48858ca08434a1329da1`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:20:56.339721 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-638ad4f1-1637-4b50-81ce-312697058ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:20:56.408667 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1260/2511] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:20:56 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:20:56.419836 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b76d9b-d02c-4407-a68e-3292e9d6b59d admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:20:56.422273 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-96b76d9b-d02c-4407-a68e-3292e9d6b59d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:20:56.479632 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b76d9b-d02c-4407-a68e-3292e9d6b59d admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:20:56.484114 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170ff827-d991-40d2-a432-ccecf46d8c83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:20:56.484764 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ff827-d991-40d2-a432-ccecf46d8c83 service cinder] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:56.485065 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ff827-d991-40d2-a432-ccecf46d8c83 service cinder] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:56.485339 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ff827-d991-40d2-a432-ccecf46d8c83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:56.494709 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170ff827-d991-40d2-a432-ccecf46d8c83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:20:56.496178 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170ff827-d991-40d2-a432-ccecf46d8c83 service cinder] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:20:56.505737 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1252/2512] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:20:56 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:20:56.582402 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ad4f1-1637-4b50-81ce-312697058ff2 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:20:56.592043 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ad4f1-1637-4b50-81ce-312697058ff2 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:20:56.596333 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ffd5a-e615-48c1-af4a-f951f603b54d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:20:56.596939 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ffd5a-e615-48c1-af4a-f951f603b54d service cinder] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:20:56.597229 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ffd5a-e615-48c1-af4a-f951f603b54d service cinder] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:20:56.597487 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ffd5a-e615-48c1-af4a-f951f603b54d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:20:56.603050 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:20:56.657629 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027ffd5a-e615-48c1-af4a-f951f603b54d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:20:56.659165 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027ffd5a-e615-48c1-af4a-f951f603b54d service cinder] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:20:56.667911 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1261/2513] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:20:56 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:21:05.544994 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc875d6f-050b-4740-8079-367693375559 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:05.545462 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc875d6f-050b-4740-8079-367693375559 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:05.545678 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc875d6f-050b-4740-8079-367693375559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:05.565077 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-dc875d6f-050b-4740-8079-367693375559 None None] MFA Rules not processed for user `755b04cae45746569bfbf92f2f213a58`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:21:05.566956 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-dc875d6f-050b-4740-8079-367693375559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:05.630572 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1253/2514] 10.222.0.61 () {58 vars in 1009 bytes} [Fri Aug 26 12:21:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:21:05.643484 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ffd5a-e615-48c1-af4a-f951f603b54d service cinder] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:05.645106 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-027ffd5a-e615-48c1-af4a-f951f603b54d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:05.714039 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ffd5a-e615-48c1-af4a-f951f603b54d service cinder] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:05.719178 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfd0cbc-441b-4d3b-ac55-c98b8572e7ac service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead95a244d11432d9fe9f718e1a75f38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:05.719807 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd0cbc-441b-4d3b-ac55-c98b8572e7ac service swift] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:05.720081 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd0cbc-441b-4d3b-ac55-c98b8572e7ac service swift] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:05.720460 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd0cbc-441b-4d3b-ac55-c98b8572e7ac service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:05.728547 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfd0cbc-441b-4d3b-ac55-c98b8572e7ac service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:05.730123 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfd0cbc-441b-4d3b-ac55-c98b8572e7ac service swift] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:05.739326 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1262/2515] 10.222.0.61 () {60 vars in 1384 bytes} [Fri Aug 26 12:21:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 12:21:12.254618 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc875d6f-050b-4740-8079-367693375559 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:12.371053 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc875d6f-050b-4740-8079-367693375559 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:12.377535 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb489ad-d3a3-4cba-a517-b688a05c9ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:12.378208 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb489ad-d3a3-4cba-a517-b688a05c9ce8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:12.378487 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb489ad-d3a3-4cba-a517-b688a05c9ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:12.379186 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb489ad-d3a3-4cba-a517-b688a05c9ce8 admin admin] PATH_INFO: `/v3/users/98c7d230233c4743aa8fccd885f1f04e` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:12.380400 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb489ad-d3a3-4cba-a517-b688a05c9ce8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98c7d230233c4743aa8fccd885f1f04e)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:12.382983 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb489ad-d3a3-4cba-a517-b688a05c9ce8 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:12.448317 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:INSERT=1,keystone:SELECT=93,keystone:DELETE=6 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:21:12.455165 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-7cb489ad-d3a3-4cba-a517-b688a05c9ce8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98c7d230233c4743aa8fccd885f1f04e'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:12.468915 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-7cb489ad-d3a3-4cba-a517-b688a05c9ce8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98c7d230233c4743aa8fccd885f1f04e'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:12.480867 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-7cb489ad-d3a3-4cba-a517-b688a05c9ce8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98c7d230233c4743aa8fccd885f1f04e'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:12.482013 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-7cb489ad-d3a3-4cba-a517-b688a05c9ce8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98c7d230233c4743aa8fccd885f1f04e'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:12.496622 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-7cb489ad-d3a3-4cba-a517-b688a05c9ce8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98c7d230233c4743aa8fccd885f1f04e'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:12.507255 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1254/2516] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:21:12 2022] DELETE /identity/v3/users/98c7d230233c4743aa8fccd885f1f04e => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:21:12.512053 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f26d44-628e-43a9-957e-5bfff8e3fc4a None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:12.512954 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f26d44-628e-43a9-957e-5bfff8e3fc4a None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:12.513393 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f26d44-628e-43a9-957e-5bfff8e3fc4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:12.539952 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-a7f26d44-628e-43a9-957e-5bfff8e3fc4a None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:21:12.541213 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-a7f26d44-628e-43a9-957e-5bfff8e3fc4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:12.627051 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1263/2517] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:21:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:21:12.635453 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb489ad-d3a3-4cba-a517-b688a05c9ce8 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:12.637143 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-7cb489ad-d3a3-4cba-a517-b688a05c9ce8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:12.707909 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb489ad-d3a3-4cba-a517-b688a05c9ce8 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:12.711550 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12930870-f7df-4e9e-904d-bbde4d619d06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:12.712159 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12930870-f7df-4e9e-904d-bbde4d619d06 service neutron] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:12.712433 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12930870-f7df-4e9e-904d-bbde4d619d06 service neutron] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:12.712708 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12930870-f7df-4e9e-904d-bbde4d619d06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:12.723248 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12930870-f7df-4e9e-904d-bbde4d619d06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:12.724690 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12930870-f7df-4e9e-904d-bbde4d619d06 service neutron] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:12.736766 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1255/2518] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:21:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:21:13.054546 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f26d44-628e-43a9-957e-5bfff8e3fc4a None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:13.056206 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-a7f26d44-628e-43a9-957e-5bfff8e3fc4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:13.099876 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f26d44-628e-43a9-957e-5bfff8e3fc4a None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:13.112110 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fcd3ab-db3d-4993-9688-2d4b90fed6ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:13.113380 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fcd3ab-db3d-4993-9688-2d4b90fed6ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:13.113924 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fcd3ab-db3d-4993-9688-2d4b90fed6ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:13.114647 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fcd3ab-db3d-4993-9688-2d4b90fed6ea admin admin] PATH_INFO: `/v3/projects/ab431fd0cf9549d6b24bc49984628214` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:13.116119 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fcd3ab-db3d-4993-9688-2d4b90fed6ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab431fd0cf9549d6b24bc49984628214)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:13.120595 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fcd3ab-db3d-4993-9688-2d4b90fed6ea admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:13.186085 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:21:13.208175 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-c0fcd3ab-db3d-4993-9688-2d4b90fed6ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab431fd0cf9549d6b24bc49984628214', 'request_id': 'req-c0fcd3ab-db3d-4993-9688-2d4b90fed6ea'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:13.219813 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-c0fcd3ab-db3d-4993-9688-2d4b90fed6ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab431fd0cf9549d6b24bc49984628214', 'request_id': 'req-c0fcd3ab-db3d-4993-9688-2d4b90fed6ea'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:13.234306 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-c0fcd3ab-db3d-4993-9688-2d4b90fed6ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab431fd0cf9549d6b24bc49984628214', 'request_id': 'req-c0fcd3ab-db3d-4993-9688-2d4b90fed6ea'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:13.267703 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-c0fcd3ab-db3d-4993-9688-2d4b90fed6ea admin admin] The token cache is being invalidate because project ab431fd0cf9549d6b24bc49984628214 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81712) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:21:13.268565 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-c0fcd3ab-db3d-4993-9688-2d4b90fed6ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:13.269556 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-c0fcd3ab-db3d-4993-9688-2d4b90fed6ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:13.271734 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1264/2519] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:21:13 2022] DELETE /identity/v3/projects/ab431fd0cf9549d6b24bc49984628214 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:21:13.277166 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12930870-f7df-4e9e-904d-bbde4d619d06 service neutron] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:13.288873 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12930870-f7df-4e9e-904d-bbde4d619d06 service neutron] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:13.358929 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7943104e-508b-4d10-90a8-0a70d92ac67c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:13.359791 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7943104e-508b-4d10-90a8-0a70d92ac67c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:13.360258 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7943104e-508b-4d10-90a8-0a70d92ac67c admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:13.360602 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7943104e-508b-4d10-90a8-0a70d92ac67c admin admin] PATH_INFO: `/v3/users/ec26eeb74e7b48858ca08434a1329da1` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:13.362083 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7943104e-508b-4d10-90a8-0a70d92ac67c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec26eeb74e7b48858ca08434a1329da1)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:13.364576 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7943104e-508b-4d10-90a8-0a70d92ac67c admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:13.436313 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-7943104e-508b-4d10-90a8-0a70d92ac67c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec26eeb74e7b48858ca08434a1329da1'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:13.446610 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-7943104e-508b-4d10-90a8-0a70d92ac67c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec26eeb74e7b48858ca08434a1329da1'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:13.464030 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-7943104e-508b-4d10-90a8-0a70d92ac67c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec26eeb74e7b48858ca08434a1329da1'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:13.465063 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-7943104e-508b-4d10-90a8-0a70d92ac67c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec26eeb74e7b48858ca08434a1329da1'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:13.468020 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:21:13.483835 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-7943104e-508b-4d10-90a8-0a70d92ac67c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec26eeb74e7b48858ca08434a1329da1'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:13.491278 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1256/2520] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:21:13 2022] DELETE /identity/v3/users/ec26eeb74e7b48858ca08434a1329da1 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:21:13.865373 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fcd3ab-db3d-4993-9688-2d4b90fed6ea admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:13.866976 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-c0fcd3ab-db3d-4993-9688-2d4b90fed6ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:13.882435 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fcd3ab-db3d-4993-9688-2d4b90fed6ea admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:13.959475 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747c3ac2-4473-45cd-98b6-342adb53e683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:13.960439 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c3ac2-4473-45cd-98b6-342adb53e683 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:13.960907 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c3ac2-4473-45cd-98b6-342adb53e683 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:13.961217 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c3ac2-4473-45cd-98b6-342adb53e683 admin admin] PATH_INFO: `/v3/projects/3905fc2662ad402fb9621844541a6f06` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:13.962603 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747c3ac2-4473-45cd-98b6-342adb53e683 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3905fc2662ad402fb9621844541a6f06)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:13.964633 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747c3ac2-4473-45cd-98b6-342adb53e683 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:14.016901 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-747c3ac2-4473-45cd-98b6-342adb53e683 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3905fc2662ad402fb9621844541a6f06', 'request_id': 'req-747c3ac2-4473-45cd-98b6-342adb53e683'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:14.023713 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-747c3ac2-4473-45cd-98b6-342adb53e683 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3905fc2662ad402fb9621844541a6f06', 'request_id': 'req-747c3ac2-4473-45cd-98b6-342adb53e683'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:14.036543 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-747c3ac2-4473-45cd-98b6-342adb53e683 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3905fc2662ad402fb9621844541a6f06', 'request_id': 'req-747c3ac2-4473-45cd-98b6-342adb53e683'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:14.050135 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-747c3ac2-4473-45cd-98b6-342adb53e683 admin admin] The token cache is being invalidate because project 3905fc2662ad402fb9621844541a6f06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81712) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:21:14.050594 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-747c3ac2-4473-45cd-98b6-342adb53e683 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:14.051496 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-747c3ac2-4473-45cd-98b6-342adb53e683 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:14.056547 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1265/2521] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:21:13 2022] DELETE /identity/v3/projects/3905fc2662ad402fb9621844541a6f06 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:21:14.058152 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7640e6-ec78-41ec-b24d-d95a1c813fad None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:14.058451 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7640e6-ec78-41ec-b24d-d95a1c813fad None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:14.058730 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7640e6-ec78-41ec-b24d-d95a1c813fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:14.083261 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-1b7640e6-ec78-41ec-b24d-d95a1c813fad None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:21:14.084549 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-1b7640e6-ec78-41ec-b24d-d95a1c813fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:14.163952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1257/2522] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:21:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:21:14.173481 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d89eb-290d-4798-a952-b3eb290a9cf2 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:14.174328 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d89eb-290d-4798-a952-b3eb290a9cf2 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:14.174614 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d89eb-290d-4798-a952-b3eb290a9cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:14.198548 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-250d89eb-290d-4798-a952-b3eb290a9cf2 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:21:14.199803 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-250d89eb-290d-4798-a952-b3eb290a9cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:14.217944 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1266/2523] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:21:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:21:14.219072 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7640e6-ec78-41ec-b24d-d95a1c813fad None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:14.231421 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7640e6-ec78-41ec-b24d-d95a1c813fad None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:14.236623 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c439aa4-8325-462e-bd0f-b12269705d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:14.237242 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c439aa4-8325-462e-bd0f-b12269705d77 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:14.237523 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c439aa4-8325-462e-bd0f-b12269705d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:14.237799 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c439aa4-8325-462e-bd0f-b12269705d77 admin admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:14.238702 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c439aa4-8325-462e-bd0f-b12269705d77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:21:14.239008 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c439aa4-8325-462e-bd0f-b12269705d77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:14.240685 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c439aa4-8325-462e-bd0f-b12269705d77 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:14.256606 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1258/2524] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:21:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:21:14.261700 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250d89eb-290d-4798-a952-b3eb290a9cf2 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:14.273917 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250d89eb-290d-4798-a952-b3eb290a9cf2 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:14.278704 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1678fd-729a-44ae-988e-33cc86d24996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:14.279974 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1678fd-729a-44ae-988e-33cc86d24996 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:14.280259 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1678fd-729a-44ae-988e-33cc86d24996 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:14.280534 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1678fd-729a-44ae-988e-33cc86d24996 admin admin] PATH_INFO: `/v3/projects` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:14.281169 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1678fd-729a-44ae-988e-33cc86d24996 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:14.283059 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1678fd-729a-44ae-988e-33cc86d24996 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:14.340442 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1267/2525] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:21:14 2022] POST /identity/v3/projects => generated 363 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 12:21:14.344867 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c439aa4-8325-462e-bd0f-b12269705d77 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:14.357122 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c439aa4-8325-462e-bd0f-b12269705d77 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:14.468484 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bab923-1364-437f-a83b-b5e06580067d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:14.469444 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bab923-1364-437f-a83b-b5e06580067d admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:14.469444 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bab923-1364-437f-a83b-b5e06580067d admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:14.469444 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bab923-1364-437f-a83b-b5e06580067d admin admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:14.470904 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bab923-1364-437f-a83b-b5e06580067d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:14.472572 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bab923-1364-437f-a83b-b5e06580067d admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:14.499429 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1259/2526] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:21:14 2022] POST /identity/v3/users => generated 282 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:21:14.503690 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1678fd-729a-44ae-988e-33cc86d24996 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:14.516953 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1678fd-729a-44ae-988e-33cc86d24996 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:14.521907 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ec55c0-78c4-4248-96f3-f909ee755669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:14.522502 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec55c0-78c4-4248-96f3-f909ee755669 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:14.522782 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec55c0-78c4-4248-96f3-f909ee755669 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:14.523055 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec55c0-78c4-4248-96f3-f909ee755669 admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:14.523734 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ec55c0-78c4-4248-96f3-f909ee755669 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:14.526564 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ec55c0-78c4-4248-96f3-f909ee755669 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:14.539183 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1268/2527] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:21:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:21:14.543100 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bab923-1364-437f-a83b-b5e06580067d admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:14.557188 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bab923-1364-437f-a83b-b5e06580067d admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:14.563477 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd81fb12-edeb-4baf-b762-f60f7cee3777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:14.564075 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81fb12-edeb-4baf-b762-f60f7cee3777 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:14.564357 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81fb12-edeb-4baf-b762-f60f7cee3777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:14.564626 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81fb12-edeb-4baf-b762-f60f7cee3777 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:14.565650 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd81fb12-edeb-4baf-b762-f60f7cee3777 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:14.568899 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd81fb12-edeb-4baf-b762-f60f7cee3777 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:14.584710 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1260/2528] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:21:14 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:21:14.586608 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ec55c0-78c4-4248-96f3-f909ee755669 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:14.598411 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ec55c0-78c4-4248-96f3-f909ee755669 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:14.603044 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb82209-f687-4d4e-b22a-5ae30d7fbf84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:14.603731 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb82209-f687-4d4e-b22a-5ae30d7fbf84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:14.604010 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb82209-f687-4d4e-b22a-5ae30d7fbf84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:14.604284 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb82209-f687-4d4e-b22a-5ae30d7fbf84 admin admin] PATH_INFO: `/v3/projects/3eb8cc00711e46a394d8d8db1238f535/users/c058c8d8a57045f6bdefc9502b81a1ac/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:14.617131 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:21:14.630410 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb82209-f687-4d4e-b22a-5ae30d7fbf84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3eb8cc00711e46a394d8d8db1238f535, user_id=c058c8d8a57045f6bdefc9502b81a1ac, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:14.633501 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb82209-f687-4d4e-b22a-5ae30d7fbf84 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:14.663233 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1269/2529] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:21:14 2022] PUT /identity/v3/projects/3eb8cc00711e46a394d8d8db1238f535/users/c058c8d8a57045f6bdefc9502b81a1ac/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:21:14.670924 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ffc995-ad09-4ef6-ac4d-a878ffc89190 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:14.671618 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ffc995-ad09-4ef6-ac4d-a878ffc89190 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:14.671944 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ffc995-ad09-4ef6-ac4d-a878ffc89190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:14.725910 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-76ffc995-ad09-4ef6-ac4d-a878ffc89190 None None] MFA Rules not processed for user `c058c8d8a57045f6bdefc9502b81a1ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:21:14.727132 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-76ffc995-ad09-4ef6-ac4d-a878ffc89190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:14.799622 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:21:14.846976 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1261/2530] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:21:14 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:21:14.855412 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb82209-f687-4d4e-b22a-5ae30d7fbf84 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:14.948093 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb82209-f687-4d4e-b22a-5ae30d7fbf84 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:14.965173 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042be3a1-fe2a-4fc4-954c-f75f8c93d979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:14.965873 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042be3a1-fe2a-4fc4-954c-f75f8c93d979 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:14.966161 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042be3a1-fe2a-4fc4-954c-f75f8c93d979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:14.967800 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042be3a1-fe2a-4fc4-954c-f75f8c93d979 admin admin] PATH_INFO: `/v3/projects` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:14.969037 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042be3a1-fe2a-4fc4-954c-f75f8c93d979 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:14.971723 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042be3a1-fe2a-4fc4-954c-f75f8c93d979 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:15.072623 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1270/2531] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:21:14 2022] POST /identity/v3/projects => generated 363 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 12:21:15.074210 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ffc995-ad09-4ef6-ac4d-a878ffc89190 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:15.086023 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ffc995-ad09-4ef6-ac4d-a878ffc89190 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:15.158816 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb825b60-2f3f-4f71-a7ad-e801379cf7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:15.160396 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb825b60-2f3f-4f71-a7ad-e801379cf7d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:15.160877 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb825b60-2f3f-4f71-a7ad-e801379cf7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:15.161175 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb825b60-2f3f-4f71-a7ad-e801379cf7d7 admin admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:15.161942 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb825b60-2f3f-4f71-a7ad-e801379cf7d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:15.164505 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb825b60-2f3f-4f71-a7ad-e801379cf7d7 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:15.192220 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1262/2532] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:21:15 2022] POST /identity/v3/users => generated 288 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:21:15.195610 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042be3a1-fe2a-4fc4-954c-f75f8c93d979 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:15.206943 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042be3a1-fe2a-4fc4-954c-f75f8c93d979 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:15.217962 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfb9c2f-97ba-4f81-9093-543ee86acd93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:15.218737 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfb9c2f-97ba-4f81-9093-543ee86acd93 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:15.219027 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfb9c2f-97ba-4f81-9093-543ee86acd93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:15.219603 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfb9c2f-97ba-4f81-9093-543ee86acd93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:15.223414 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfb9c2f-97ba-4f81-9093-543ee86acd93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:15.223414 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfb9c2f-97ba-4f81-9093-543ee86acd93 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:15.235241 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1271/2533] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:21:15 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:21:15.238531 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb825b60-2f3f-4f71-a7ad-e801379cf7d7 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:15.250976 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb825b60-2f3f-4f71-a7ad-e801379cf7d7 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:15.256283 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43516702-6511-4d49-b04c-90d276ed544c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:15.257105 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43516702-6511-4d49-b04c-90d276ed544c admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:15.257384 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43516702-6511-4d49-b04c-90d276ed544c admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:15.257653 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43516702-6511-4d49-b04c-90d276ed544c admin admin] PATH_INFO: `/v3/projects/a6ba2900b72347afb1f8a2c01a92ee28/users/4022fff6f5524fb4adfdf603f8bb196b/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:15.275501 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43516702-6511-4d49-b04c-90d276ed544c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6ba2900b72347afb1f8a2c01a92ee28, user_id=4022fff6f5524fb4adfdf603f8bb196b, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:15.278140 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43516702-6511-4d49-b04c-90d276ed544c admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:15.294987 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1263/2534] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:21:15 2022] PUT /identity/v3/projects/a6ba2900b72347afb1f8a2c01a92ee28/users/4022fff6f5524fb4adfdf603f8bb196b/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:21:15.299414 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfb9c2f-97ba-4f81-9093-543ee86acd93 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:15.311128 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfb9c2f-97ba-4f81-9093-543ee86acd93 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:15.383272 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebaa275-36b1-49f3-8785-04a3f9443bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:15.383272 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebaa275-36b1-49f3-8785-04a3f9443bd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:15.383272 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebaa275-36b1-49f3-8785-04a3f9443bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:15.383272 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebaa275-36b1-49f3-8785-04a3f9443bd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:15.383778 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebaa275-36b1-49f3-8785-04a3f9443bd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:15.384635 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebaa275-36b1-49f3-8785-04a3f9443bd0 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:15.395827 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1272/2535] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:21:15 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:21:15.399844 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43516702-6511-4d49-b04c-90d276ed544c admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:15.411053 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43516702-6511-4d49-b04c-90d276ed544c admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:15.416040 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c907c7f4-9ce6-4afb-b334-ff2a57611387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:15.416689 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907c7f4-9ce6-4afb-b334-ff2a57611387 admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:15.416966 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907c7f4-9ce6-4afb-b334-ff2a57611387 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:15.417229 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907c7f4-9ce6-4afb-b334-ff2a57611387 admin admin] PATH_INFO: `/v3/projects/a6ba2900b72347afb1f8a2c01a92ee28/users/4022fff6f5524fb4adfdf603f8bb196b/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:15.419334 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c907c7f4-9ce6-4afb-b334-ff2a57611387 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6ba2900b72347afb1f8a2c01a92ee28, user_id=4022fff6f5524fb4adfdf603f8bb196b, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:15.422243 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c907c7f4-9ce6-4afb-b334-ff2a57611387 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:15.433811 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1264/2536] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:21:15 2022] PUT /identity/v3/projects/a6ba2900b72347afb1f8a2c01a92ee28/users/4022fff6f5524fb4adfdf603f8bb196b/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:21:15.445475 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6846153-0c12-4603-b93c-725ec055da19 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:15.445809 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6846153-0c12-4603-b93c-725ec055da19 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:15.446022 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6846153-0c12-4603-b93c-725ec055da19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:15.478320 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-f6846153-0c12-4603-b93c-725ec055da19 None None] MFA Rules not processed for user `4022fff6f5524fb4adfdf603f8bb196b`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:21:15.479908 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-f6846153-0c12-4603-b93c-725ec055da19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:15.541313 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:21:15.595190 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1273/2537] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:21:15 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:21:15.596179 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c907c7f4-9ce6-4afb-b334-ff2a57611387 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:15.598431 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-c907c7f4-9ce6-4afb-b334-ff2a57611387 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:15.677356 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c907c7f4-9ce6-4afb-b334-ff2a57611387 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:15.682047 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607f2c28-25fe-411b-8e47-fd9a6783099b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:15.682815 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f2c28-25fe-411b-8e47-fd9a6783099b service cinder] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:15.683242 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f2c28-25fe-411b-8e47-fd9a6783099b service cinder] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:15.683599 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f2c28-25fe-411b-8e47-fd9a6783099b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:15.695986 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607f2c28-25fe-411b-8e47-fd9a6783099b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:15.697610 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607f2c28-25fe-411b-8e47-fd9a6783099b service cinder] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:15.710888 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1265/2538] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:21:15 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 12:21:15.823352 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6846153-0c12-4603-b93c-725ec055da19 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:15.841468 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6846153-0c12-4603-b93c-725ec055da19 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:15.846161 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd06780-762d-48ec-bb27-8d8d03c5a14c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:15.847139 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd06780-762d-48ec-bb27-8d8d03c5a14c service cinder] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:15.847591 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd06780-762d-48ec-bb27-8d8d03c5a14c service cinder] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:15.847864 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd06780-762d-48ec-bb27-8d8d03c5a14c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:15.915835 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd06780-762d-48ec-bb27-8d8d03c5a14c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:15.917359 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd06780-762d-48ec-bb27-8d8d03c5a14c service cinder] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:15.931962 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1274/2539] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:21:15 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:21:16.123863 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607f2c28-25fe-411b-8e47-fd9a6783099b service cinder] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:16.125299 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-607f2c28-25fe-411b-8e47-fd9a6783099b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:16.160135 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:21:16.182209 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607f2c28-25fe-411b-8e47-fd9a6783099b service cinder] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:16.185949 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc110efb-866a-442e-aace-54ea3d91a7f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:16.186551 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc110efb-866a-442e-aace-54ea3d91a7f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:16.186829 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc110efb-866a-442e-aace-54ea3d91a7f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:16.187096 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc110efb-866a-442e-aace-54ea3d91a7f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:16.188836 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-dc110efb-866a-442e-aace-54ea3d91a7f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:16.251276 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc110efb-866a-442e-aace-54ea3d91a7f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:16.252644 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc110efb-866a-442e-aace-54ea3d91a7f7 service neutron] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:16.262407 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1266/2540] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:21:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:21:18.716380 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd06780-762d-48ec-bb27-8d8d03c5a14c service cinder] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:18.729127 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd06780-762d-48ec-bb27-8d8d03c5a14c service cinder] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:18.737978 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728a71c0-f4c1-4345-aa5c-b442fe000f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:18.738872 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a71c0-f4c1-4345-aa5c-b442fe000f25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:18.739152 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a71c0-f4c1-4345-aa5c-b442fe000f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:18.739564 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a71c0-f4c1-4345-aa5c-b442fe000f25 admin admin] PATH_INFO: `/v3/users/c058c8d8a57045f6bdefc9502b81a1ac` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:18.740708 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728a71c0-f4c1-4345-aa5c-b442fe000f25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c058c8d8a57045f6bdefc9502b81a1ac)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:18.742635 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728a71c0-f4c1-4345-aa5c-b442fe000f25 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:18.804743 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-728a71c0-f4c1-4345-aa5c-b442fe000f25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c058c8d8a57045f6bdefc9502b81a1ac'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:18.812392 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-728a71c0-f4c1-4345-aa5c-b442fe000f25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c058c8d8a57045f6bdefc9502b81a1ac'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:18.813285 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-728a71c0-f4c1-4345-aa5c-b442fe000f25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c058c8d8a57045f6bdefc9502b81a1ac'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:18.820698 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-728a71c0-f4c1-4345-aa5c-b442fe000f25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c058c8d8a57045f6bdefc9502b81a1ac'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:18.821413 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-728a71c0-f4c1-4345-aa5c-b442fe000f25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c058c8d8a57045f6bdefc9502b81a1ac'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:18.841375 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1275/2541] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:21:18 2022] DELETE /identity/v3/users/c058c8d8a57045f6bdefc9502b81a1ac => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:21:18.846526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be07b6f-284b-4b86-b5b9-f7aa0128d516 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:18.847035 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be07b6f-284b-4b86-b5b9-f7aa0128d516 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:18.847157 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be07b6f-284b-4b86-b5b9-f7aa0128d516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:18.867153 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-4be07b6f-284b-4b86-b5b9-f7aa0128d516 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:21:18.868427 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-4be07b6f-284b-4b86-b5b9-f7aa0128d516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:18.934119 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1267/2542] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:21:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:21:18.945270 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728a71c0-f4c1-4345-aa5c-b442fe000f25 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:18.946758 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-728a71c0-f4c1-4345-aa5c-b442fe000f25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:19.019442 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728a71c0-f4c1-4345-aa5c-b442fe000f25 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:19.024459 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa59c3a-b43c-4d56-9519-5c936aa9fb53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:19.025085 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa59c3a-b43c-4d56-9519-5c936aa9fb53 service neutron] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:19.025367 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa59c3a-b43c-4d56-9519-5c936aa9fb53 service neutron] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:19.025638 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa59c3a-b43c-4d56-9519-5c936aa9fb53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:19.033057 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:21:19.037371 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa59c3a-b43c-4d56-9519-5c936aa9fb53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:19.038887 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa59c3a-b43c-4d56-9519-5c936aa9fb53 service neutron] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:19.050938 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1276/2543] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:21:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:21:19.377575 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be07b6f-284b-4b86-b5b9-f7aa0128d516 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:19.379463 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-4be07b6f-284b-4b86-b5b9-f7aa0128d516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:19.395476 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be07b6f-284b-4b86-b5b9-f7aa0128d516 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:19.402207 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dfda87-7468-4764-9ee2-884e912e5843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:19.403420 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dfda87-7468-4764-9ee2-884e912e5843 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:19.403879 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dfda87-7468-4764-9ee2-884e912e5843 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:19.404469 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dfda87-7468-4764-9ee2-884e912e5843 admin admin] PATH_INFO: `/v3/projects/3eb8cc00711e46a394d8d8db1238f535` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:19.405961 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dfda87-7468-4764-9ee2-884e912e5843 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3eb8cc00711e46a394d8d8db1238f535)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:19.407943 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dfda87-7468-4764-9ee2-884e912e5843 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:19.553971 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-84dfda87-7468-4764-9ee2-884e912e5843 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3eb8cc00711e46a394d8d8db1238f535', 'request_id': 'req-84dfda87-7468-4764-9ee2-884e912e5843'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:19.561384 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-84dfda87-7468-4764-9ee2-884e912e5843 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3eb8cc00711e46a394d8d8db1238f535', 'request_id': 'req-84dfda87-7468-4764-9ee2-884e912e5843'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:19.575246 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-84dfda87-7468-4764-9ee2-884e912e5843 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3eb8cc00711e46a394d8d8db1238f535', 'request_id': 'req-84dfda87-7468-4764-9ee2-884e912e5843'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:19.590231 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-84dfda87-7468-4764-9ee2-884e912e5843 admin admin] The token cache is being invalidate because project 3eb8cc00711e46a394d8d8db1238f535 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:21:19.590678 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-84dfda87-7468-4764-9ee2-884e912e5843 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:19.592675 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-84dfda87-7468-4764-9ee2-884e912e5843 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:19.594888 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1268/2544] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:21:19 2022] DELETE /identity/v3/projects/3eb8cc00711e46a394d8d8db1238f535 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:21:19.599160 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa59c3a-b43c-4d56-9519-5c936aa9fb53 service neutron] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:19.610547 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa59c3a-b43c-4d56-9519-5c936aa9fb53 service neutron] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:19.709990 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716d1b14-b25a-434c-aa9c-f62158eef4e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:19.710658 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716d1b14-b25a-434c-aa9c-f62158eef4e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:19.710940 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716d1b14-b25a-434c-aa9c-f62158eef4e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:19.711907 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716d1b14-b25a-434c-aa9c-f62158eef4e9 admin admin] PATH_INFO: `/v3/users/4022fff6f5524fb4adfdf603f8bb196b` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:19.713157 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716d1b14-b25a-434c-aa9c-f62158eef4e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4022fff6f5524fb4adfdf603f8bb196b)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:19.715698 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716d1b14-b25a-434c-aa9c-f62158eef4e9 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:19.786719 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-716d1b14-b25a-434c-aa9c-f62158eef4e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4022fff6f5524fb4adfdf603f8bb196b'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:19.797743 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-716d1b14-b25a-434c-aa9c-f62158eef4e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4022fff6f5524fb4adfdf603f8bb196b'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:19.798430 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-716d1b14-b25a-434c-aa9c-f62158eef4e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4022fff6f5524fb4adfdf603f8bb196b'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:19.807127 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-716d1b14-b25a-434c-aa9c-f62158eef4e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4022fff6f5524fb4adfdf603f8bb196b'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:19.809195 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-716d1b14-b25a-434c-aa9c-f62158eef4e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4022fff6f5524fb4adfdf603f8bb196b'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:19.840712 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1277/2545] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:21:19 2022] DELETE /identity/v3/users/4022fff6f5524fb4adfdf603f8bb196b => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:21:20.277460 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dfda87-7468-4764-9ee2-884e912e5843 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:20.279503 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-84dfda87-7468-4764-9ee2-884e912e5843 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:20.293665 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dfda87-7468-4764-9ee2-884e912e5843 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:20.299788 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:21:20.337753 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716d1b14-b25a-434c-aa9c-f62158eef4e9 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:20.339312 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-716d1b14-b25a-434c-aa9c-f62158eef4e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:20.373554 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9172606a-1ee7-43cb-ada2-3acbcf5691fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:20.374209 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9172606a-1ee7-43cb-ada2-3acbcf5691fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:20.374758 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9172606a-1ee7-43cb-ada2-3acbcf5691fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:20.375025 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9172606a-1ee7-43cb-ada2-3acbcf5691fd admin admin] PATH_INFO: `/v3/users/36a27ca98a2e4521b269be48c9ec4cb0` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:20.387482 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9172606a-1ee7-43cb-ada2-3acbcf5691fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=36a27ca98a2e4521b269be48c9ec4cb0)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:20.387482 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9172606a-1ee7-43cb-ada2-3acbcf5691fd admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:20.431675 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-9172606a-1ee7-43cb-ada2-3acbcf5691fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36a27ca98a2e4521b269be48c9ec4cb0'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:20.435074 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716d1b14-b25a-434c-aa9c-f62158eef4e9 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:20.451468 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-9172606a-1ee7-43cb-ada2-3acbcf5691fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36a27ca98a2e4521b269be48c9ec4cb0'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:20.451593 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5f4de1-ad23-4e75-bae8-0b27ec98bc78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:20.451593 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5f4de1-ad23-4e75-bae8-0b27ec98bc78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:20.451593 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5f4de1-ad23-4e75-bae8-0b27ec98bc78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:20.451593 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5f4de1-ad23-4e75-bae8-0b27ec98bc78 admin admin] PATH_INFO: `/v3/projects/a6ba2900b72347afb1f8a2c01a92ee28` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:20.451969 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5f4de1-ad23-4e75-bae8-0b27ec98bc78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6ba2900b72347afb1f8a2c01a92ee28)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:20.451969 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5f4de1-ad23-4e75-bae8-0b27ec98bc78 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:20.452145 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-9172606a-1ee7-43cb-ada2-3acbcf5691fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36a27ca98a2e4521b269be48c9ec4cb0'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:20.452145 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-9172606a-1ee7-43cb-ada2-3acbcf5691fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36a27ca98a2e4521b269be48c9ec4cb0'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:20.462952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-9172606a-1ee7-43cb-ada2-3acbcf5691fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36a27ca98a2e4521b269be48c9ec4cb0'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:20.467644 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:21:20.471840 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1269/2546] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:21:20 2022] DELETE /identity/v3/users/36a27ca98a2e4521b269be48c9ec4cb0 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:21:20.516583 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-df5f4de1-ad23-4e75-bae8-0b27ec98bc78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6ba2900b72347afb1f8a2c01a92ee28', 'request_id': 'req-df5f4de1-ad23-4e75-bae8-0b27ec98bc78'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:20.526568 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-df5f4de1-ad23-4e75-bae8-0b27ec98bc78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6ba2900b72347afb1f8a2c01a92ee28', 'request_id': 'req-df5f4de1-ad23-4e75-bae8-0b27ec98bc78'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:20.536657 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-df5f4de1-ad23-4e75-bae8-0b27ec98bc78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6ba2900b72347afb1f8a2c01a92ee28', 'request_id': 'req-df5f4de1-ad23-4e75-bae8-0b27ec98bc78'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:20.549893 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-df5f4de1-ad23-4e75-bae8-0b27ec98bc78 admin admin] The token cache is being invalidate because project a6ba2900b72347afb1f8a2c01a92ee28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81712) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:21:20.550651 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-df5f4de1-ad23-4e75-bae8-0b27ec98bc78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:20.551125 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-df5f4de1-ad23-4e75-bae8-0b27ec98bc78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:20.552976 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1278/2547] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:21:20 2022] DELETE /identity/v3/projects/a6ba2900b72347afb1f8a2c01a92ee28 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 12:21:20.557761 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58be600c-d040-4b75-8c98-bfe6d30418aa None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:20.558198 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58be600c-d040-4b75-8c98-bfe6d30418aa None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:20.558509 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58be600c-d040-4b75-8c98-bfe6d30418aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:20.582982 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-58be600c-d040-4b75-8c98-bfe6d30418aa None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:21:20.584083 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-58be600c-d040-4b75-8c98-bfe6d30418aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:20.650004 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc04d4a-4154-4b0b-9ab7-bc779053bb36 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:20.650264 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc04d4a-4154-4b0b-9ab7-bc779053bb36 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:20.654748 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc04d4a-4154-4b0b-9ab7-bc779053bb36 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:20.658952 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1279/2548] 10.222.0.61 () {54 vars in 921 bytes} [Fri Aug 26 12:21:20 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:21:20.665633 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b118ba-00cf-469e-a666-c952bcf12265 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:20.666032 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b118ba-00cf-469e-a666-c952bcf12265 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:20.666289 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b118ba-00cf-469e-a666-c952bcf12265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:20.683435 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1270/2549] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:21:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:21:20.688204 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58be600c-d040-4b75-8c98-bfe6d30418aa None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:20.689762 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-58be600c-d040-4b75-8c98-bfe6d30418aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:20.693306 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-25b118ba-00cf-469e-a666-c952bcf12265 None None] MFA Rules not processed for user `7aacd7d6d17e4f10ba5c9107a12beafa`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:21:20.694883 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-25b118ba-00cf-469e-a666-c952bcf12265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:20.713665 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58be600c-d040-4b75-8c98-bfe6d30418aa None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:20.723977 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3933062-b753-4198-a0e9-4e1bdcd29746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:20.724684 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3933062-b753-4198-a0e9-4e1bdcd29746 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:20.724968 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3933062-b753-4198-a0e9-4e1bdcd29746 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:20.725287 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3933062-b753-4198-a0e9-4e1bdcd29746 admin admin] PATH_INFO: `/v3/projects/9574496fe7ba4144a05fca690cc3486f` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:20.726648 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3933062-b753-4198-a0e9-4e1bdcd29746 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9574496fe7ba4144a05fca690cc3486f)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:20.728960 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3933062-b753-4198-a0e9-4e1bdcd29746 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:20.770204 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:21:20.780940 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1280/2550] 10.222.0.61 () {58 vars in 1018 bytes} [Fri Aug 26 12:21:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:21:20.782216 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d8c30-fd5f-4fe1-822d-a478c0185ee7 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:20.782523 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d8c30-fd5f-4fe1-822d-a478c0185ee7 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:20.782780 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d8c30-fd5f-4fe1-822d-a478c0185ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:20.804564 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-495d8c30-fd5f-4fe1-822d-a478c0185ee7 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:21:20.805820 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-495d8c30-fd5f-4fe1-822d-a478c0185ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:20.806869 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-e3933062-b753-4198-a0e9-4e1bdcd29746 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9574496fe7ba4144a05fca690cc3486f', 'request_id': 'req-e3933062-b753-4198-a0e9-4e1bdcd29746'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:20.814200 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-e3933062-b753-4198-a0e9-4e1bdcd29746 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9574496fe7ba4144a05fca690cc3486f', 'request_id': 'req-e3933062-b753-4198-a0e9-4e1bdcd29746'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:20.821757 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-e3933062-b753-4198-a0e9-4e1bdcd29746 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9574496fe7ba4144a05fca690cc3486f', 'request_id': 'req-e3933062-b753-4198-a0e9-4e1bdcd29746'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:20.833128 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-e3933062-b753-4198-a0e9-4e1bdcd29746 admin admin] The token cache is being invalidate because project 9574496fe7ba4144a05fca690cc3486f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:21:20.833417 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-e3933062-b753-4198-a0e9-4e1bdcd29746 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:20.834116 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-e3933062-b753-4198-a0e9-4e1bdcd29746 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:20.836258 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1271/2551] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:21:20 2022] DELETE /identity/v3/projects/9574496fe7ba4144a05fca690cc3486f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:21:20.836788 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3933062-b753-4198-a0e9-4e1bdcd29746 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:20.842658 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-e3933062-b753-4198-a0e9-4e1bdcd29746 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:20.869822 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1281/2552] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:21:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:21:20.871062 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f6562-9b55-43e6-be43-ba459bcde7ae None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:20.871375 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f6562-9b55-43e6-be43-ba459bcde7ae None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:20.871649 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f6562-9b55-43e6-be43-ba459bcde7ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:20.893423 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-7b8f6562-9b55-43e6-be43-ba459bcde7ae None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:21:20.894664 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-7b8f6562-9b55-43e6-be43-ba459bcde7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:20.903191 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1282/2553] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:21:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:21:20.903407 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3933062-b753-4198-a0e9-4e1bdcd29746 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:20.903934 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8f6562-9b55-43e6-be43-ba459bcde7ae None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:20.912846 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd25344-1eb0-4dff-b7dd-60faeb556b21 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152fd7c708114f249b9c960165f0fdfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:20.913487 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd25344-1eb0-4dff-b7dd-60faeb556b21 service glance] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:20.913768 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd25344-1eb0-4dff-b7dd-60faeb556b21 service glance] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:20.914107 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8f6562-9b55-43e6-be43-ba459bcde7ae None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:20.914226 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd25344-1eb0-4dff-b7dd-60faeb556b21 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:20.916059 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-8dd25344-1eb0-4dff-b7dd-60faeb556b21 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:20.918909 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36f3683-8718-4202-b1d7-9f3d554966cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:20.919540 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f3683-8718-4202-b1d7-9f3d554966cf admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:20.919865 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f3683-8718-4202-b1d7-9f3d554966cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:20.920142 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f3683-8718-4202-b1d7-9f3d554966cf admin admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:20.920900 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36f3683-8718-4202-b1d7-9f3d554966cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:21:20.921210 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36f3683-8718-4202-b1d7-9f3d554966cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:20.922834 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36f3683-8718-4202-b1d7-9f3d554966cf admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:20.934564 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1283/2554] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:21:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:21:20.935796 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6276adcf-750d-48fc-b28a-8eb8a43cc8ba None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:20.936079 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6276adcf-750d-48fc-b28a-8eb8a43cc8ba None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:20.936357 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6276adcf-750d-48fc-b28a-8eb8a43cc8ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:20.955515 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-6276adcf-750d-48fc-b28a-8eb8a43cc8ba None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:21:20.956742 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-6276adcf-750d-48fc-b28a-8eb8a43cc8ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:20.964988 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1284/2555] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:21:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:21:20.965742 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6276adcf-750d-48fc-b28a-8eb8a43cc8ba None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:20.975540 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6276adcf-750d-48fc-b28a-8eb8a43cc8ba None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:20.980089 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d7278d-c453-472b-ad94-a01920f2dafc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:20.980660 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d7278d-c453-472b-ad94-a01920f2dafc admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:20.980943 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d7278d-c453-472b-ad94-a01920f2dafc admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:20.981215 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d7278d-c453-472b-ad94-a01920f2dafc admin admin] PATH_INFO: `/v3/projects` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:20.981828 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d7278d-c453-472b-ad94-a01920f2dafc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:20.983817 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d7278d-c453-472b-ad94-a01920f2dafc admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:20.991412 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd25344-1eb0-4dff-b7dd-60faeb556b21 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:20.992842 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd25344-1eb0-4dff-b7dd-60faeb556b21 service glance] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:20.993596 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:21:21.004915 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1272/2556] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:21:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:21:21.006118 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd25344-1eb0-4dff-b7dd-60faeb556b21 service glance] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:21.018414 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1285/2557] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:21:20 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:21:21.018609 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd25344-1eb0-4dff-b7dd-60faeb556b21 service glance] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:21.020193 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d7278d-c453-472b-ad94-a01920f2dafc admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:21.077419 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:21:21.089542 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7521202b-dcfe-4716-aedc-d8d5462ea45a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:21.090149 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7521202b-dcfe-4716-aedc-d8d5462ea45a admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:21.090427 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7521202b-dcfe-4716-aedc-d8d5462ea45a admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:21.090739 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7521202b-dcfe-4716-aedc-d8d5462ea45a admin admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:21.091444 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7521202b-dcfe-4716-aedc-d8d5462ea45a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:21:21.091778 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7521202b-dcfe-4716-aedc-d8d5462ea45a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:21.093505 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7521202b-dcfe-4716-aedc-d8d5462ea45a admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:21.099876 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d7278d-c453-472b-ad94-a01920f2dafc admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:21.104606 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdbf9ca-b1ec-47d3-b6c2-9e8eb24f1841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:21.105199 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdbf9ca-b1ec-47d3-b6c2-9e8eb24f1841 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:21.109260 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdbf9ca-b1ec-47d3-b6c2-9e8eb24f1841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:21.109402 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1273/2558] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:21:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:21:21.109509 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdbf9ca-b1ec-47d3-b6c2-9e8eb24f1841 admin admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:21.109509 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdbf9ca-b1ec-47d3-b6c2-9e8eb24f1841 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:21.109689 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d9ab36-2f29-449b-a708-1a008ce3cdf9 None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:21.109689 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d9ab36-2f29-449b-a708-1a008ce3cdf9 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:21.109689 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d9ab36-2f29-449b-a708-1a008ce3cdf9 None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:21.109935 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdbf9ca-b1ec-47d3-b6c2-9e8eb24f1841 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:21.110030 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1274/2559] 10.222.0.61 () {54 vars in 921 bytes} [Fri Aug 26 12:21:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:21:21.112093 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d9ab36-2f29-449b-a708-1a008ce3cdf9 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:21.126745 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d9ab36-2f29-449b-a708-1a008ce3cdf9 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:21.131624 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17e71d8-a4b5-4493-98ce-ab04e4237904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:21.134707 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1286/2560] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:21:21 2022] POST /identity/v3/users => generated 293 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:21:21.134707 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0d6a2-3a66-43af-8a5c-82fb1a2f3214 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:21.134707 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0d6a2-3a66-43af-8a5c-82fb1a2f3214 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:21.134707 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0d6a2-3a66-43af-8a5c-82fb1a2f3214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:21.135759 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e71d8-a4b5-4493-98ce-ab04e4237904 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:21.136047 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e71d8-a4b5-4493-98ce-ab04e4237904 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:21.136328 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e71d8-a4b5-4493-98ce-ab04e4237904 admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:21.136956 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17e71d8-a4b5-4493-98ce-ab04e4237904 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:21.139499 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17e71d8-a4b5-4493-98ce-ab04e4237904 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:21.154990 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-c6f0d6a2-3a66-43af-8a5c-82fb1a2f3214 None None] MFA Rules not processed for user `b0972dbd5cff4610b239326425e6351c`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:21:21.156170 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-c6f0d6a2-3a66-43af-8a5c-82fb1a2f3214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:21.167234 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1275/2561] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:21:21 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:21:21.168200 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17e71d8-a4b5-4493-98ce-ab04e4237904 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:21.213002 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1287/2562] 10.222.0.61 () {58 vars in 1018 bytes} [Fri Aug 26 12:21:21 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:21:21.214366 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f0d6a2-3a66-43af-8a5c-82fb1a2f3214 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:21.226098 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f0d6a2-3a66-43af-8a5c-82fb1a2f3214 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:21.230729 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b0cd8b-a78a-4981-bc94-d0dffd7addee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:21.231437 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0cd8b-a78a-4981-bc94-d0dffd7addee admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:21.231798 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0cd8b-a78a-4981-bc94-d0dffd7addee admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:21.232070 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17e71d8-a4b5-4493-98ce-ab04e4237904 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:21.232283 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0cd8b-a78a-4981-bc94-d0dffd7addee admin admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:21.233136 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b0cd8b-a78a-4981-bc94-d0dffd7addee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:21.235112 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b0cd8b-a78a-4981-bc94-d0dffd7addee admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:21.236824 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7946a4b1-4941-464f-b2ed-533d8187c796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:21.237452 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7946a4b1-4941-464f-b2ed-533d8187c796 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:21.237709 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7946a4b1-4941-464f-b2ed-533d8187c796 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:21.237984 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7946a4b1-4941-464f-b2ed-533d8187c796 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:21.238905 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7946a4b1-4941-464f-b2ed-533d8187c796 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:21.240526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7946a4b1-4941-464f-b2ed-533d8187c796 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:21.251927 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1276/2563] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:21:21.252789 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7946a4b1-4941-464f-b2ed-533d8187c796 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:21.254771 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-7946a4b1-4941-464f-b2ed-533d8187c796 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:21.258610 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1288/2564] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:21:21 2022] POST /identity/v3/users => generated 293 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:21:21.259859 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b0cd8b-a78a-4981-bc94-d0dffd7addee admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:21.271147 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b0cd8b-a78a-4981-bc94-d0dffd7addee admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:21.275832 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af6aa4b-4a89-4e86-ae0b-9ba037327366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:21.276420 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af6aa4b-4a89-4e86-ae0b-9ba037327366 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:21.276701 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af6aa4b-4a89-4e86-ae0b-9ba037327366 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:21.276971 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af6aa4b-4a89-4e86-ae0b-9ba037327366 admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:21.277607 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af6aa4b-4a89-4e86-ae0b-9ba037327366 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:21.279245 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af6aa4b-4a89-4e86-ae0b-9ba037327366 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:21.289881 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1289/2565] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:21:21.293271 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af6aa4b-4a89-4e86-ae0b-9ba037327366 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:21.300248 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af6aa4b-4a89-4e86-ae0b-9ba037327366 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:21.306161 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6502e0-11e7-4094-93bb-1f900aff8ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:21.306774 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6502e0-11e7-4094-93bb-1f900aff8ac6 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:21.307051 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6502e0-11e7-4094-93bb-1f900aff8ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:21.307354 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6502e0-11e7-4094-93bb-1f900aff8ac6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:21.307978 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6502e0-11e7-4094-93bb-1f900aff8ac6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:21.309540 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6502e0-11e7-4094-93bb-1f900aff8ac6 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:21.315383 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7946a4b1-4941-464f-b2ed-533d8187c796 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:21.319061 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a9efc6-264f-4306-a3c9-492f34b60eb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:21.319689 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9efc6-264f-4306-a3c9-492f34b60eb0 service neutron] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:21.319970 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9efc6-264f-4306-a3c9-492f34b60eb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:21.320265 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9efc6-264f-4306-a3c9-492f34b60eb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:21.320784 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1290/2566] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:21:21.321690 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6502e0-11e7-4094-93bb-1f900aff8ac6 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:21.329340 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a9efc6-264f-4306-a3c9-492f34b60eb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:21.330845 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a9efc6-264f-4306-a3c9-492f34b60eb0 service neutron] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:21.332793 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6502e0-11e7-4094-93bb-1f900aff8ac6 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:21.337403 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2cb5a2-1e2d-4d64-84fe-042c0a7e3c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:21.338043 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2cb5a2-1e2d-4d64-84fe-042c0a7e3c36 admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:21.338311 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2cb5a2-1e2d-4d64-84fe-042c0a7e3c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:21.338580 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2cb5a2-1e2d-4d64-84fe-042c0a7e3c36 admin admin] PATH_INFO: `/v3/projects/28eeeb8e3e58438184756c2f8547fd01/users/81c68228d75f462dac0a8ff148a3fcfe/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:21.339023 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1277/2567] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:21:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:21:21.339896 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a9efc6-264f-4306-a3c9-492f34b60eb0 service neutron] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:21.351254 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a9efc6-264f-4306-a3c9-492f34b60eb0 service neutron] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:21.356084 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c1cfea-e918-4ae7-9f6e-d537fbaf1bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:21.356273 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2cb5a2-1e2d-4d64-84fe-042c0a7e3c36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28eeeb8e3e58438184756c2f8547fd01, user_id=81c68228d75f462dac0a8ff148a3fcfe, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:21.356764 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c1cfea-e918-4ae7-9f6e-d537fbaf1bd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:21.357047 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c1cfea-e918-4ae7-9f6e-d537fbaf1bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:21.357514 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c1cfea-e918-4ae7-9f6e-d537fbaf1bd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:21.358646 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c1cfea-e918-4ae7-9f6e-d537fbaf1bd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:21.359050 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2cb5a2-1e2d-4d64-84fe-042c0a7e3c36 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:21.360460 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c1cfea-e918-4ae7-9f6e-d537fbaf1bd3 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:21.373605 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1278/2568] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:21:21.386240 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1291/2569] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:21:21 2022] PUT /identity/v3/projects/28eeeb8e3e58438184756c2f8547fd01/users/81c68228d75f462dac0a8ff148a3fcfe/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:21:21.386564 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c1cfea-e918-4ae7-9f6e-d537fbaf1bd3 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:21.387845 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7e4523-187a-4220-a375-a8767f0ad154 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:21.388138 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7e4523-187a-4220-a375-a8767f0ad154 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:21.388420 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7e4523-187a-4220-a375-a8767f0ad154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:21.399500 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c1cfea-e918-4ae7-9f6e-d537fbaf1bd3 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:21.421291 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-ed7e4523-187a-4220-a375-a8767f0ad154 None None] MFA Rules not processed for user `81c68228d75f462dac0a8ff148a3fcfe`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:21:21.422577 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-ed7e4523-187a-4220-a375-a8767f0ad154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:21.437150 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:21:21.448839 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:21:21.461359 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f37fea0-95db-42a6-bfcb-bc099bd43f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:21.462021 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f37fea0-95db-42a6-bfcb-bc099bd43f4e admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:21.462303 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f37fea0-95db-42a6-bfcb-bc099bd43f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:21.462581 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f37fea0-95db-42a6-bfcb-bc099bd43f4e admin admin] PATH_INFO: `/v3/projects/5ff2ea99f18f449a87c6b4625110527f/users/3893122ee89f4bb3af4d7dbd478b22d8/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:21.478025 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f37fea0-95db-42a6-bfcb-bc099bd43f4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ff2ea99f18f449a87c6b4625110527f, user_id=3893122ee89f4bb3af4d7dbd478b22d8, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:21.480703 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f37fea0-95db-42a6-bfcb-bc099bd43f4e admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:21.492417 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1292/2570] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:21:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:21:21.494730 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1279/2571] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:21:21 2022] PUT /identity/v3/projects/5ff2ea99f18f449a87c6b4625110527f/users/3893122ee89f4bb3af4d7dbd478b22d8/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:21:21.497112 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8f9d50-bba4-4850-9ceb-532ff2951ce7 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:21.497408 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8f9d50-bba4-4850-9ceb-532ff2951ce7 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:21.497701 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8f9d50-bba4-4850-9ceb-532ff2951ce7 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:21.499613 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1293/2572] 10.222.0.61 () {54 vars in 921 bytes} [Fri Aug 26 12:21:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:21:21.504173 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f37fea0-95db-42a6-bfcb-bc099bd43f4e admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:21.505701 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b4420b-ef83-44c7-9905-55120eeb624d None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:21.505981 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b4420b-ef83-44c7-9905-55120eeb624d None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:21.506257 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b4420b-ef83-44c7-9905-55120eeb624d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:21.525931 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-e2b4420b-ef83-44c7-9905-55120eeb624d None None] MFA Rules not processed for user `7aacd7d6d17e4f10ba5c9107a12beafa`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:21:21.527124 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-e2b4420b-ef83-44c7-9905-55120eeb624d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:21.568027 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f37fea0-95db-42a6-bfcb-bc099bd43f4e admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:21.572821 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e95499-e03a-4428-b609-d20fbf779b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:21.573436 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e95499-e03a-4428-b609-d20fbf779b4b admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:21.573745 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e95499-e03a-4428-b609-d20fbf779b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:21.574019 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e95499-e03a-4428-b609-d20fbf779b4b admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:21.574652 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e95499-e03a-4428-b609-d20fbf779b4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:21.576704 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e95499-e03a-4428-b609-d20fbf779b4b admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:21.594953 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1294/2573] 10.222.0.61 () {58 vars in 1018 bytes} [Fri Aug 26 12:21:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:21:21.601457 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e504d023-9a24-41b3-bf6c-e28bd12e5716 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:21.601898 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e504d023-9a24-41b3-bf6c-e28bd12e5716 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:21.602264 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e504d023-9a24-41b3-bf6c-e28bd12e5716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:21.603430 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1280/2574] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:21:21 2022] POST /identity/v3/projects => generated 387 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 12:21:21.608160 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e95499-e03a-4428-b609-d20fbf779b4b admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:21.632738 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-e504d023-9a24-41b3-bf6c-e28bd12e5716 None None] MFA Rules not processed for user `3893122ee89f4bb3af4d7dbd478b22d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:21:21.634203 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-e504d023-9a24-41b3-bf6c-e28bd12e5716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:21.672774 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e95499-e03a-4428-b609-d20fbf779b4b admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:21.677418 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe3d9af-2537-474e-8fb4-5facfa74fe86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:21.678028 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe3d9af-2537-474e-8fb4-5facfa74fe86 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:21.678315 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe3d9af-2537-474e-8fb4-5facfa74fe86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:21.678593 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe3d9af-2537-474e-8fb4-5facfa74fe86 admin admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:21.679266 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe3d9af-2537-474e-8fb4-5facfa74fe86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:21.681225 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe3d9af-2537-474e-8fb4-5facfa74fe86 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:21.699810 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1295/2575] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:21:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:21:21.700647 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e504d023-9a24-41b3-bf6c-e28bd12e5716 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:21.706246 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1281/2576] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:21:21 2022] POST /identity/v3/users => generated 294 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:21:21.707262 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe3d9af-2537-474e-8fb4-5facfa74fe86 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:21.718005 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe3d9af-2537-474e-8fb4-5facfa74fe86 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:21.722846 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5f71b1-b889-433c-b1b5-16d9527f149f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:21.723467 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f71b1-b889-433c-b1b5-16d9527f149f admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:21.723746 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f71b1-b889-433c-b1b5-16d9527f149f admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:21.724021 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f71b1-b889-433c-b1b5-16d9527f149f admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:21.724655 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5f71b1-b889-433c-b1b5-16d9527f149f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:21.726224 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5f71b1-b889-433c-b1b5-16d9527f149f admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:21.738318 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1282/2577] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:21:21.738318 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5f71b1-b889-433c-b1b5-16d9527f149f admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:21.743333 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-ed5f71b1-b889-433c-b1b5-16d9527f149f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:21.747032 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:21:21.766099 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e504d023-9a24-41b3-bf6c-e28bd12e5716 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:21.770408 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf693c9-8630-4c16-87c5-06709d989382 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:21.771011 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf693c9-8630-4c16-87c5-06709d989382 service neutron] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:21.771318 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf693c9-8630-4c16-87c5-06709d989382 service neutron] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:21.771591 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf693c9-8630-4c16-87c5-06709d989382 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:21.789284 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:21:21.828251 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5f71b1-b889-433c-b1b5-16d9527f149f admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:21.836294 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f8fcd7-e3bf-4504-9f1d-6a9f581dd8c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aacd7d6d17e4f10ba5c9107a12beafa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:21.836917 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f8fcd7-e3bf-4504-9f1d-6a9f581dd8c2 service nova] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:21.837203 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f8fcd7-e3bf-4504-9f1d-6a9f581dd8c2 service nova] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:21.837481 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f8fcd7-e3bf-4504-9f1d-6a9f581dd8c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:21.843281 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf693c9-8630-4c16-87c5-06709d989382 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:21.846103 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf693c9-8630-4c16-87c5-06709d989382 service neutron] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:21.847372 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f8fcd7-e3bf-4504-9f1d-6a9f581dd8c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:21.848966 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f8fcd7-e3bf-4504-9f1d-6a9f581dd8c2 service nova] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:21.858555 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1296/2578] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:21:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:21:21.865048 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf693c9-8630-4c16-87c5-06709d989382 service neutron] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:21.871412 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1283/2579] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:21:21 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:21:21.878255 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf693c9-8630-4c16-87c5-06709d989382 service neutron] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:21.883107 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fc87bb-1c7e-4033-b44f-b77e15876398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:21.883750 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc87bb-1c7e-4033-b44f-b77e15876398 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:21.884182 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc87bb-1c7e-4033-b44f-b77e15876398 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:21.884422 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc87bb-1c7e-4033-b44f-b77e15876398 admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:21.885173 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fc87bb-1c7e-4033-b44f-b77e15876398 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:21.886761 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fc87bb-1c7e-4033-b44f-b77e15876398 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:21.898602 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1297/2580] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:21:21.902639 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f8fcd7-e3bf-4504-9f1d-6a9f581dd8c2 service nova] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:21.918770 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f8fcd7-e3bf-4504-9f1d-6a9f581dd8c2 service nova] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:21.924541 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42465e4-9a57-45f2-afc8-58f6ff8f01f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:21.925104 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42465e4-9a57-45f2-afc8-58f6ff8f01f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:21.925389 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42465e4-9a57-45f2-afc8-58f6ff8f01f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:21.925664 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42465e4-9a57-45f2-afc8-58f6ff8f01f2 admin admin] PATH_INFO: `/v3/projects/938cfff1a00b4bd198fdbc01ecfa290c/users/45e555b98494435d80b7ca932ef2ea0e/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:21.946895 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42465e4-9a57-45f2-afc8-58f6ff8f01f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=938cfff1a00b4bd198fdbc01ecfa290c, user_id=45e555b98494435d80b7ca932ef2ea0e, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:21.950745 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42465e4-9a57-45f2-afc8-58f6ff8f01f2 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:21.969367 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1284/2581] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:21:21 2022] PUT /identity/v3/projects/938cfff1a00b4bd198fdbc01ecfa290c/users/45e555b98494435d80b7ca932ef2ea0e/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:21:21.980688 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e2e7b-c42d-4514-8878-b16c37b5ed04 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:21.981144 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e2e7b-c42d-4514-8878-b16c37b5ed04 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:21.981442 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e2e7b-c42d-4514-8878-b16c37b5ed04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:22.014186 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-099e2e7b-c42d-4514-8878-b16c37b5ed04 None None] MFA Rules not processed for user `45e555b98494435d80b7ca932ef2ea0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:21:22.015595 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-099e2e7b-c42d-4514-8878-b16c37b5ed04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:22.103257 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1298/2582] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:21:21 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:21:22.105085 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42465e4-9a57-45f2-afc8-58f6ff8f01f2 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:22.193896 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42465e4-9a57-45f2-afc8-58f6ff8f01f2 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:22.199521 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e215e5f-ce0c-4fe2-a81b-060f4fac4769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:22.200484 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e215e5f-ce0c-4fe2-a81b-060f4fac4769 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:22.200759 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e215e5f-ce0c-4fe2-a81b-060f4fac4769 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:22.201599 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e215e5f-ce0c-4fe2-a81b-060f4fac4769 admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:22.202326 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e215e5f-ce0c-4fe2-a81b-060f4fac4769 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:22.204246 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e215e5f-ce0c-4fe2-a81b-060f4fac4769 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:22.248025 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1285/2583] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:21:22 2022] POST /identity/v3/projects => generated 385 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:21:22.249829 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e2e7b-c42d-4514-8878-b16c37b5ed04 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:22.269471 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e2e7b-c42d-4514-8878-b16c37b5ed04 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:22.338559 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=100 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:21:22.348497 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090f1b47-a70e-494b-b2c9-e5d98a30e506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:22.349122 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090f1b47-a70e-494b-b2c9-e5d98a30e506 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:22.349404 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090f1b47-a70e-494b-b2c9-e5d98a30e506 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:22.349682 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090f1b47-a70e-494b-b2c9-e5d98a30e506 admin admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:22.350356 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090f1b47-a70e-494b-b2c9-e5d98a30e506 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:22.352368 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090f1b47-a70e-494b-b2c9-e5d98a30e506 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:22.381691 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1299/2584] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:21:22 2022] POST /identity/v3/users => generated 299 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:21:22.383466 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e215e5f-ce0c-4fe2-a81b-060f4fac4769 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:22.394685 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e215e5f-ce0c-4fe2-a81b-060f4fac4769 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:22.399954 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7d8d4c-b2ed-4f25-88f1-fb5ea29af339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:22.400564 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7d8d4c-b2ed-4f25-88f1-fb5ea29af339 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:22.400866 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7d8d4c-b2ed-4f25-88f1-fb5ea29af339 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:22.401432 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7d8d4c-b2ed-4f25-88f1-fb5ea29af339 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:22.402085 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7d8d4c-b2ed-4f25-88f1-fb5ea29af339 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:22.403739 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7d8d4c-b2ed-4f25-88f1-fb5ea29af339 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:22.415298 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1286/2585] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:21:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:21:22.421089 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090f1b47-a70e-494b-b2c9-e5d98a30e506 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:22.433322 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090f1b47-a70e-494b-b2c9-e5d98a30e506 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:22.440638 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8325eaf0-82e0-4cbb-a339-3528dba1354c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:22.441317 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8325eaf0-82e0-4cbb-a339-3528dba1354c admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:22.441604 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8325eaf0-82e0-4cbb-a339-3528dba1354c admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:22.442320 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8325eaf0-82e0-4cbb-a339-3528dba1354c admin admin] PATH_INFO: `/v3/projects/0746f1408db04caf938bb2b7879e57c3/users/47557612ae6141e8a22040334f9159cf/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:22.458041 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8325eaf0-82e0-4cbb-a339-3528dba1354c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0746f1408db04caf938bb2b7879e57c3, user_id=47557612ae6141e8a22040334f9159cf, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:22.460775 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8325eaf0-82e0-4cbb-a339-3528dba1354c admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:22.474143 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1300/2586] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:21:22 2022] PUT /identity/v3/projects/0746f1408db04caf938bb2b7879e57c3/users/47557612ae6141e8a22040334f9159cf/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:21:22.478379 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7d8d4c-b2ed-4f25-88f1-fb5ea29af339 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:22.489246 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7d8d4c-b2ed-4f25-88f1-fb5ea29af339 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:22.547600 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7cd7ce-cefd-4aa2-b565-32735ec0497f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:22.548625 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7cd7ce-cefd-4aa2-b565-32735ec0497f admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:22.548625 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7cd7ce-cefd-4aa2-b565-32735ec0497f admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:22.548780 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7cd7ce-cefd-4aa2-b565-32735ec0497f admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:22.549432 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7cd7ce-cefd-4aa2-b565-32735ec0497f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:22.551037 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7cd7ce-cefd-4aa2-b565-32735ec0497f admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:22.565008 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1287/2587] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:21:22 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:21:22.568237 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8325eaf0-82e0-4cbb-a339-3528dba1354c admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:22.579709 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8325eaf0-82e0-4cbb-a339-3528dba1354c admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:22.584844 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ed3ce5-1114-40d0-91c4-40761346d511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:22.585528 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed3ce5-1114-40d0-91c4-40761346d511 admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:22.585805 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed3ce5-1114-40d0-91c4-40761346d511 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:22.586074 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed3ce5-1114-40d0-91c4-40761346d511 admin admin] PATH_INFO: `/v3/projects/0746f1408db04caf938bb2b7879e57c3/users/47557612ae6141e8a22040334f9159cf/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:22.588004 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ed3ce5-1114-40d0-91c4-40761346d511 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0746f1408db04caf938bb2b7879e57c3, user_id=47557612ae6141e8a22040334f9159cf, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:22.590668 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ed3ce5-1114-40d0-91c4-40761346d511 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:22.601416 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1301/2588] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:21:22 2022] PUT /identity/v3/projects/0746f1408db04caf938bb2b7879e57c3/users/47557612ae6141e8a22040334f9159cf/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:21:22.612864 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e91b3d-d391-4eed-b9b3-6f9f48ec59ed None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:22.613513 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e91b3d-d391-4eed-b9b3-6f9f48ec59ed None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:22.613792 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e91b3d-d391-4eed-b9b3-6f9f48ec59ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:22.632709 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:21:22.642718 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-35e91b3d-d391-4eed-b9b3-6f9f48ec59ed None None] MFA Rules not processed for user `47557612ae6141e8a22040334f9159cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:21:22.644007 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-35e91b3d-d391-4eed-b9b3-6f9f48ec59ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:22.711549 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1288/2589] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:21:22 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:21:22.723044 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ed3ce5-1114-40d0-91c4-40761346d511 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:22.724766 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-c6ed3ce5-1114-40d0-91c4-40761346d511 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:22.784467 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ed3ce5-1114-40d0-91c4-40761346d511 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:22.788104 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add19ffe-4d54-4135-ac8f-7329934b1b40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:22.788703 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add19ffe-4d54-4135-ac8f-7329934b1b40 service cinder] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:22.788979 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add19ffe-4d54-4135-ac8f-7329934b1b40 service cinder] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:22.789246 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add19ffe-4d54-4135-ac8f-7329934b1b40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:22.798975 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add19ffe-4d54-4135-ac8f-7329934b1b40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:22.800400 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add19ffe-4d54-4135-ac8f-7329934b1b40 service cinder] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:22.809830 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1302/2590] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:21:22 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:21:23.350612 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e91b3d-d391-4eed-b9b3-6f9f48ec59ed None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:23.361754 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e91b3d-d391-4eed-b9b3-6f9f48ec59ed None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:23.366229 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb3720b-256b-43ea-be64-a986f146452b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:23.366759 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb3720b-256b-43ea-be64-a986f146452b service cinder] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:23.367037 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb3720b-256b-43ea-be64-a986f146452b service cinder] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:23.367343 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb3720b-256b-43ea-be64-a986f146452b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:23.425176 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb3720b-256b-43ea-be64-a986f146452b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:23.426610 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb3720b-256b-43ea-be64-a986f146452b service cinder] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:23.435778 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1289/2591] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:21:23 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:21:26.723492 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3cedc1-5a00-4448-b275-442921244a70 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:26.723892 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3cedc1-5a00-4448-b275-442921244a70 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:26.724047 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3cedc1-5a00-4448-b275-442921244a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:26.746357 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-7a3cedc1-5a00-4448-b275-442921244a70 None None] MFA Rules not processed for user `755b04cae45746569bfbf92f2f213a58`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:21:26.747489 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-7a3cedc1-5a00-4448-b275-442921244a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:26.806552 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1303/2592] 10.222.0.61 () {58 vars in 1009 bytes} [Fri Aug 26 12:21:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:21:26.818354 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb3720b-256b-43ea-be64-a986f146452b service cinder] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:26.819751 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-ccb3720b-256b-43ea-be64-a986f146452b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:26.893708 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb3720b-256b-43ea-be64-a986f146452b service cinder] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:26.900447 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07c74d3-2429-4eac-92a9-25e9bb3c4c82 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead95a244d11432d9fe9f718e1a75f38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:26.901334 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c74d3-2429-4eac-92a9-25e9bb3c4c82 service swift] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:26.901791 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c74d3-2429-4eac-92a9-25e9bb3c4c82 service swift] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:26.902413 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c74d3-2429-4eac-92a9-25e9bb3c4c82 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:26.914903 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07c74d3-2429-4eac-92a9-25e9bb3c4c82 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:26.917265 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07c74d3-2429-4eac-92a9-25e9bb3c4c82 service swift] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:26.927954 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1290/2593] 10.222.0.61 () {60 vars in 1384 bytes} [Fri Aug 26 12:21:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 12:21:29.362195 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3cedc1-5a00-4448-b275-442921244a70 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:29.434383 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:21:29.451292 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3cedc1-5a00-4448-b275-442921244a70 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:29.456119 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f6dbeb-096f-4b75-b0ff-c4159bbfae52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:29.456840 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f6dbeb-096f-4b75-b0ff-c4159bbfae52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:29.457208 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f6dbeb-096f-4b75-b0ff-c4159bbfae52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:29.457544 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f6dbeb-096f-4b75-b0ff-c4159bbfae52 admin admin] PATH_INFO: `/v3/users/3893122ee89f4bb3af4d7dbd478b22d8` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:29.458796 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f6dbeb-096f-4b75-b0ff-c4159bbfae52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3893122ee89f4bb3af4d7dbd478b22d8)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:29.460842 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f6dbeb-096f-4b75-b0ff-c4159bbfae52 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:29.517806 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-87f6dbeb-096f-4b75-b0ff-c4159bbfae52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3893122ee89f4bb3af4d7dbd478b22d8'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:29.524801 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-87f6dbeb-096f-4b75-b0ff-c4159bbfae52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3893122ee89f4bb3af4d7dbd478b22d8'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:29.525645 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-87f6dbeb-096f-4b75-b0ff-c4159bbfae52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3893122ee89f4bb3af4d7dbd478b22d8'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:29.533910 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-87f6dbeb-096f-4b75-b0ff-c4159bbfae52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3893122ee89f4bb3af4d7dbd478b22d8'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:29.534611 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-87f6dbeb-096f-4b75-b0ff-c4159bbfae52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3893122ee89f4bb3af4d7dbd478b22d8'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:29.575189 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1304/2594] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:21:29 2022] DELETE /identity/v3/users/3893122ee89f4bb3af4d7dbd478b22d8 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:21:29.579810 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e4e657-e339-4766-9407-6169e0ef8c46 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:29.580820 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e4e657-e339-4766-9407-6169e0ef8c46 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:29.581261 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e4e657-e339-4766-9407-6169e0ef8c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:29.611228 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-f8e4e657-e339-4766-9407-6169e0ef8c46 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:21:29.612474 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-f8e4e657-e339-4766-9407-6169e0ef8c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:29.632622 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:21:29.702557 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1291/2595] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:21:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:21:29.708547 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f6dbeb-096f-4b75-b0ff-c4159bbfae52 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:29.710457 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-87f6dbeb-096f-4b75-b0ff-c4159bbfae52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:29.816685 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f6dbeb-096f-4b75-b0ff-c4159bbfae52 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:29.827382 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec0989d-57cd-416f-b13a-48741e9b3a5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:29.828382 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec0989d-57cd-416f-b13a-48741e9b3a5f service neutron] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:29.828664 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec0989d-57cd-416f-b13a-48741e9b3a5f service neutron] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:29.828934 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec0989d-57cd-416f-b13a-48741e9b3a5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:29.839151 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec0989d-57cd-416f-b13a-48741e9b3a5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:29.840767 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec0989d-57cd-416f-b13a-48741e9b3a5f service neutron] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:29.859449 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1305/2596] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:21:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:21:29.863716 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e4e657-e339-4766-9407-6169e0ef8c46 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:29.865411 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-f8e4e657-e339-4766-9407-6169e0ef8c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:29.956114 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e4e657-e339-4766-9407-6169e0ef8c46 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:29.961295 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6384e2c-6624-4d6f-a0c9-a1047efa304a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:29.961894 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6384e2c-6624-4d6f-a0c9-a1047efa304a service cinder] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:29.962179 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6384e2c-6624-4d6f-a0c9-a1047efa304a service cinder] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:29.962455 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6384e2c-6624-4d6f-a0c9-a1047efa304a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:29.964560 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-b6384e2c-6624-4d6f-a0c9-a1047efa304a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:30.065216 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6384e2c-6624-4d6f-a0c9-a1047efa304a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:30.067196 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6384e2c-6624-4d6f-a0c9-a1047efa304a service cinder] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:30.079241 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1292/2597] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:21:29 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 12:21:30.203822 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec0989d-57cd-416f-b13a-48741e9b3a5f service neutron] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:30.206230 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-2ec0989d-57cd-416f-b13a-48741e9b3a5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:30.251615 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec0989d-57cd-416f-b13a-48741e9b3a5f service neutron] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:30.268884 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a6674a-118c-44eb-8e56-6db78b7ef2a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:30.269642 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a6674a-118c-44eb-8e56-6db78b7ef2a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:30.270156 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a6674a-118c-44eb-8e56-6db78b7ef2a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:30.270609 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a6674a-118c-44eb-8e56-6db78b7ef2a1 admin admin] PATH_INFO: `/v3/projects/5ff2ea99f18f449a87c6b4625110527f` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:30.272405 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a6674a-118c-44eb-8e56-6db78b7ef2a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ff2ea99f18f449a87c6b4625110527f)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:30.275127 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a6674a-118c-44eb-8e56-6db78b7ef2a1 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:30.370092 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-15a6674a-118c-44eb-8e56-6db78b7ef2a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ff2ea99f18f449a87c6b4625110527f', 'request_id': 'req-15a6674a-118c-44eb-8e56-6db78b7ef2a1'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:30.381492 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-15a6674a-118c-44eb-8e56-6db78b7ef2a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ff2ea99f18f449a87c6b4625110527f', 'request_id': 'req-15a6674a-118c-44eb-8e56-6db78b7ef2a1'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:30.396686 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-15a6674a-118c-44eb-8e56-6db78b7ef2a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ff2ea99f18f449a87c6b4625110527f', 'request_id': 'req-15a6674a-118c-44eb-8e56-6db78b7ef2a1'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:30.415640 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:21:30.422836 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-15a6674a-118c-44eb-8e56-6db78b7ef2a1 admin admin] The token cache is being invalidate because project 5ff2ea99f18f449a87c6b4625110527f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81712) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:21:30.423463 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-15a6674a-118c-44eb-8e56-6db78b7ef2a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:30.424693 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-15a6674a-118c-44eb-8e56-6db78b7ef2a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:30.427279 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1306/2598] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:21:30 2022] DELETE /identity/v3/projects/5ff2ea99f18f449a87c6b4625110527f => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:21:30.433467 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dfe8eb-d529-4ab1-990b-c9dc86e09886 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:30.433798 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dfe8eb-d529-4ab1-990b-c9dc86e09886 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:30.434018 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dfe8eb-d529-4ab1-990b-c9dc86e09886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:30.461299 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-85dfe8eb-d529-4ab1-990b-c9dc86e09886 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:21:30.462713 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-85dfe8eb-d529-4ab1-990b-c9dc86e09886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:30.541853 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1293/2599] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:21:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:21:30.551598 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b760b5-b5ff-4792-834d-314a544ba419 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:30.551881 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b760b5-b5ff-4792-834d-314a544ba419 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:30.552154 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b760b5-b5ff-4792-834d-314a544ba419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:30.576090 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-b6b760b5-b5ff-4792-834d-314a544ba419 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:21:30.577364 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-b6b760b5-b5ff-4792-834d-314a544ba419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:30.586041 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1307/2600] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:21:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:21:30.590483 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dfe8eb-d529-4ab1-990b-c9dc86e09886 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:30.601162 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dfe8eb-d529-4ab1-990b-c9dc86e09886 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:30.607954 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994da7d4-0a05-4ca1-bf2d-a59ba76cc75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:30.610236 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994da7d4-0a05-4ca1-bf2d-a59ba76cc75f admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:30.610531 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994da7d4-0a05-4ca1-bf2d-a59ba76cc75f admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:30.610804 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994da7d4-0a05-4ca1-bf2d-a59ba76cc75f admin admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:30.611519 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994da7d4-0a05-4ca1-bf2d-a59ba76cc75f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:21:30.611840 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994da7d4-0a05-4ca1-bf2d-a59ba76cc75f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:30.617259 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994da7d4-0a05-4ca1-bf2d-a59ba76cc75f admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:30.648824 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1294/2601] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:21:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:21:30.653264 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b760b5-b5ff-4792-834d-314a544ba419 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:30.666368 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b760b5-b5ff-4792-834d-314a544ba419 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:30.679918 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5989aa7b-f63a-4b60-9e36-73656836db43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:30.680882 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5989aa7b-f63a-4b60-9e36-73656836db43 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:30.680882 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5989aa7b-f63a-4b60-9e36-73656836db43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:30.681159 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5989aa7b-f63a-4b60-9e36-73656836db43 admin admin] PATH_INFO: `/v3/projects` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:30.681842 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5989aa7b-f63a-4b60-9e36-73656836db43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:30.683895 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5989aa7b-f63a-4b60-9e36-73656836db43 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:30.740526 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1308/2602] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:21:30 2022] POST /identity/v3/projects => generated 373 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:21:30.745600 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994da7d4-0a05-4ca1-bf2d-a59ba76cc75f admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:30.756676 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:21:30.761688 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994da7d4-0a05-4ca1-bf2d-a59ba76cc75f admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:30.861512 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b689066-d1f7-42ab-95f3-cb1a028e2431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:30.862452 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b689066-d1f7-42ab-95f3-cb1a028e2431 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:30.862915 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b689066-d1f7-42ab-95f3-cb1a028e2431 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:30.863639 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b689066-d1f7-42ab-95f3-cb1a028e2431 admin admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:30.864992 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b689066-d1f7-42ab-95f3-cb1a028e2431 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:30.867102 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b689066-d1f7-42ab-95f3-cb1a028e2431 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:30.900498 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1295/2603] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:21:30 2022] POST /identity/v3/users => generated 287 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:21:30.902045 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5989aa7b-f63a-4b60-9e36-73656836db43 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:30.913765 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5989aa7b-f63a-4b60-9e36-73656836db43 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:30.918501 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec94787-019d-4d2e-9798-fec9e4507692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:30.919103 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec94787-019d-4d2e-9798-fec9e4507692 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:30.919412 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec94787-019d-4d2e-9798-fec9e4507692 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:30.919771 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec94787-019d-4d2e-9798-fec9e4507692 admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:30.920510 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec94787-019d-4d2e-9798-fec9e4507692 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:30.923083 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec94787-019d-4d2e-9798-fec9e4507692 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:30.939173 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1309/2604] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:21:30 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:21:30.940531 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b689066-d1f7-42ab-95f3-cb1a028e2431 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:30.951690 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b689066-d1f7-42ab-95f3-cb1a028e2431 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:30.959894 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27521f9-b6ce-44cc-99d4-08d915796adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:30.960538 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27521f9-b6ce-44cc-99d4-08d915796adf admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:30.960815 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27521f9-b6ce-44cc-99d4-08d915796adf admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:30.961125 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27521f9-b6ce-44cc-99d4-08d915796adf admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:30.961787 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27521f9-b6ce-44cc-99d4-08d915796adf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:30.964216 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27521f9-b6ce-44cc-99d4-08d915796adf admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:30.983811 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1296/2605] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:21:30 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:21:30.986052 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec94787-019d-4d2e-9798-fec9e4507692 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:30.997205 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec94787-019d-4d2e-9798-fec9e4507692 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:31.002432 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d034ae4-2629-4df0-b63c-cf7013822dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:31.003491 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d034ae4-2629-4df0-b63c-cf7013822dd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:31.003846 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d034ae4-2629-4df0-b63c-cf7013822dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:31.004190 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d034ae4-2629-4df0-b63c-cf7013822dd3 admin admin] PATH_INFO: `/v3/projects/a895db96a8a14bb1b5e058700f5d3915/users/03624d67fc254b0c98fa42e28969d8f1/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:31.026955 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d034ae4-2629-4df0-b63c-cf7013822dd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a895db96a8a14bb1b5e058700f5d3915, user_id=03624d67fc254b0c98fa42e28969d8f1, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:31.030700 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d034ae4-2629-4df0-b63c-cf7013822dd3 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:31.055229 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1310/2606] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:21:30 2022] PUT /identity/v3/projects/a895db96a8a14bb1b5e058700f5d3915/users/03624d67fc254b0c98fa42e28969d8f1/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:21:31.065619 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27de083-1bfa-4014-83cb-2ea6634e7e31 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:31.065949 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27de083-1bfa-4014-83cb-2ea6634e7e31 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:31.066310 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27de083-1bfa-4014-83cb-2ea6634e7e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:31.105589 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-b27de083-1bfa-4014-83cb-2ea6634e7e31 None None] MFA Rules not processed for user `03624d67fc254b0c98fa42e28969d8f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:21:31.106819 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-b27de083-1bfa-4014-83cb-2ea6634e7e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:31.217035 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1297/2607] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:21:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:21:31.229460 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d034ae4-2629-4df0-b63c-cf7013822dd3 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:31.231499 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-6d034ae4-2629-4df0-b63c-cf7013822dd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:31.329073 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d034ae4-2629-4df0-b63c-cf7013822dd3 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:31.338369 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bad6a20-fa78-47cf-ba19-e0b2ece88e0e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aacd7d6d17e4f10ba5c9107a12beafa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:31.338995 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bad6a20-fa78-47cf-ba19-e0b2ece88e0e service nova] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:31.339320 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bad6a20-fa78-47cf-ba19-e0b2ece88e0e service nova] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:31.339593 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bad6a20-fa78-47cf-ba19-e0b2ece88e0e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:31.349326 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bad6a20-fa78-47cf-ba19-e0b2ece88e0e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:31.351294 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bad6a20-fa78-47cf-ba19-e0b2ece88e0e service nova] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:31.362025 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1311/2608] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:21:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:21:32.760207 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27de083-1bfa-4014-83cb-2ea6634e7e31 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:32.763999 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-b27de083-1bfa-4014-83cb-2ea6634e7e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:32.882675 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27de083-1bfa-4014-83cb-2ea6634e7e31 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:32.888721 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf0bed6-0855-430f-945c-97be14fd1205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:32.890138 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf0bed6-0855-430f-945c-97be14fd1205 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:32.890615 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf0bed6-0855-430f-945c-97be14fd1205 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:32.891176 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf0bed6-0855-430f-945c-97be14fd1205 admin admin] PATH_INFO: `/v3/users/81c68228d75f462dac0a8ff148a3fcfe` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:32.892595 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf0bed6-0855-430f-945c-97be14fd1205 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81c68228d75f462dac0a8ff148a3fcfe)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:32.900079 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf0bed6-0855-430f-945c-97be14fd1205 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:32.927496 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:21:32.992760 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-bcf0bed6-0855-430f-945c-97be14fd1205 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81c68228d75f462dac0a8ff148a3fcfe'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:33.003140 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-bcf0bed6-0855-430f-945c-97be14fd1205 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81c68228d75f462dac0a8ff148a3fcfe'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:33.012511 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-bcf0bed6-0855-430f-945c-97be14fd1205 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81c68228d75f462dac0a8ff148a3fcfe'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:33.013387 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-bcf0bed6-0855-430f-945c-97be14fd1205 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81c68228d75f462dac0a8ff148a3fcfe'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:33.032925 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-bcf0bed6-0855-430f-945c-97be14fd1205 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81c68228d75f462dac0a8ff148a3fcfe'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:33.039886 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1298/2609] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:21:32 2022] DELETE /identity/v3/users/81c68228d75f462dac0a8ff148a3fcfe => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:21:33.044063 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ddaeb9-42da-4f1c-a6d9-3a94f6fbd05c None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:33.044588 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ddaeb9-42da-4f1c-a6d9-3a94f6fbd05c None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:33.044588 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ddaeb9-42da-4f1c-a6d9-3a94f6fbd05c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:33.076329 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-92ddaeb9-42da-4f1c-a6d9-3a94f6fbd05c None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:21:33.078080 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-92ddaeb9-42da-4f1c-a6d9-3a94f6fbd05c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:33.164481 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1312/2610] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:21:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:21:33.171334 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf0bed6-0855-430f-945c-97be14fd1205 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:33.173316 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-bcf0bed6-0855-430f-945c-97be14fd1205 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:33.272049 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf0bed6-0855-430f-945c-97be14fd1205 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:33.275709 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa1314a-f8f9-4294-b5b7-bb9e1dcd7102 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:33.276314 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa1314a-f8f9-4294-b5b7-bb9e1dcd7102 service neutron] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:33.276591 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa1314a-f8f9-4294-b5b7-bb9e1dcd7102 service neutron] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:33.276859 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa1314a-f8f9-4294-b5b7-bb9e1dcd7102 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:33.289404 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa1314a-f8f9-4294-b5b7-bb9e1dcd7102 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:33.291171 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa1314a-f8f9-4294-b5b7-bb9e1dcd7102 service neutron] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:33.302258 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1299/2611] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:21:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:21:33.618296 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ddaeb9-42da-4f1c-a6d9-3a94f6fbd05c None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:33.620351 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-92ddaeb9-42da-4f1c-a6d9-3a94f6fbd05c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:33.638874 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ddaeb9-42da-4f1c-a6d9-3a94f6fbd05c None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:33.644508 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe2ab5f-19bb-4f37-9777-d4800f2d5552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:33.645150 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe2ab5f-19bb-4f37-9777-d4800f2d5552 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:33.645428 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe2ab5f-19bb-4f37-9777-d4800f2d5552 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:33.645700 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe2ab5f-19bb-4f37-9777-d4800f2d5552 admin admin] PATH_INFO: `/v3/projects/28eeeb8e3e58438184756c2f8547fd01` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:33.646825 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe2ab5f-19bb-4f37-9777-d4800f2d5552 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28eeeb8e3e58438184756c2f8547fd01)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:33.648850 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe2ab5f-19bb-4f37-9777-d4800f2d5552 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:33.675461 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:21:33.726583 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-efe2ab5f-19bb-4f37-9777-d4800f2d5552 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28eeeb8e3e58438184756c2f8547fd01', 'request_id': 'req-efe2ab5f-19bb-4f37-9777-d4800f2d5552'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:33.733838 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-efe2ab5f-19bb-4f37-9777-d4800f2d5552 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28eeeb8e3e58438184756c2f8547fd01', 'request_id': 'req-efe2ab5f-19bb-4f37-9777-d4800f2d5552'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:33.743909 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-efe2ab5f-19bb-4f37-9777-d4800f2d5552 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28eeeb8e3e58438184756c2f8547fd01', 'request_id': 'req-efe2ab5f-19bb-4f37-9777-d4800f2d5552'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:33.760711 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-efe2ab5f-19bb-4f37-9777-d4800f2d5552 admin admin] The token cache is being invalidate because project 28eeeb8e3e58438184756c2f8547fd01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81712) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:21:33.761089 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-efe2ab5f-19bb-4f37-9777-d4800f2d5552 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:33.761691 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-efe2ab5f-19bb-4f37-9777-d4800f2d5552 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:33.763432 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1313/2612] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:21:33 2022] DELETE /identity/v3/projects/28eeeb8e3e58438184756c2f8547fd01 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:21:33.767505 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa1314a-f8f9-4294-b5b7-bb9e1dcd7102 service neutron] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:33.778647 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa1314a-f8f9-4294-b5b7-bb9e1dcd7102 service neutron] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:33.840539 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c7cf60-a1e6-4303-8cae-929e24e6c66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:33.841209 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c7cf60-a1e6-4303-8cae-929e24e6c66e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:33.841493 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c7cf60-a1e6-4303-8cae-929e24e6c66e admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:33.841768 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c7cf60-a1e6-4303-8cae-929e24e6c66e admin admin] PATH_INFO: `/v3/users/45e555b98494435d80b7ca932ef2ea0e` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:33.843790 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c7cf60-a1e6-4303-8cae-929e24e6c66e admin admin] RBAC: Authorizing `identity:delete_user(user_id=45e555b98494435d80b7ca932ef2ea0e)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:33.845545 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c7cf60-a1e6-4303-8cae-929e24e6c66e admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:33.889866 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:21:33.899048 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-61c7cf60-a1e6-4303-8cae-929e24e6c66e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45e555b98494435d80b7ca932ef2ea0e'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:33.909018 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-61c7cf60-a1e6-4303-8cae-929e24e6c66e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45e555b98494435d80b7ca932ef2ea0e'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:33.916759 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-61c7cf60-a1e6-4303-8cae-929e24e6c66e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45e555b98494435d80b7ca932ef2ea0e'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:33.917440 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-61c7cf60-a1e6-4303-8cae-929e24e6c66e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45e555b98494435d80b7ca932ef2ea0e'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:33.933227 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-61c7cf60-a1e6-4303-8cae-929e24e6c66e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45e555b98494435d80b7ca932ef2ea0e'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:33.940762 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1300/2613] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:21:33 2022] DELETE /identity/v3/users/45e555b98494435d80b7ca932ef2ea0e => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:21:34.340362 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe2ab5f-19bb-4f37-9777-d4800f2d5552 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:34.343242 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-efe2ab5f-19bb-4f37-9777-d4800f2d5552 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:34.362638 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe2ab5f-19bb-4f37-9777-d4800f2d5552 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:34.395478 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c7cf60-a1e6-4303-8cae-929e24e6c66e admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:34.397641 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-61c7cf60-a1e6-4303-8cae-929e24e6c66e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:34.454375 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fe9561-de6d-462e-8803-5bd1b96d9c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:34.455238 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe9561-de6d-462e-8803-5bd1b96d9c12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:34.456860 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe9561-de6d-462e-8803-5bd1b96d9c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:34.457157 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe9561-de6d-462e-8803-5bd1b96d9c12 admin admin] PATH_INFO: `/v3/users/03624d67fc254b0c98fa42e28969d8f1` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:34.458499 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fe9561-de6d-462e-8803-5bd1b96d9c12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03624d67fc254b0c98fa42e28969d8f1)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:34.460782 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fe9561-de6d-462e-8803-5bd1b96d9c12 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:34.481835 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c7cf60-a1e6-4303-8cae-929e24e6c66e admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:34.487887 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f852c909-e63f-45f5-93b1-46a0344bc9f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:34.489057 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f852c909-e63f-45f5-93b1-46a0344bc9f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:34.489696 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f852c909-e63f-45f5-93b1-46a0344bc9f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:34.490210 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f852c909-e63f-45f5-93b1-46a0344bc9f1 admin admin] PATH_INFO: `/v3/projects/938cfff1a00b4bd198fdbc01ecfa290c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:34.491602 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f852c909-e63f-45f5-93b1-46a0344bc9f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=938cfff1a00b4bd198fdbc01ecfa290c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:34.493716 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f852c909-e63f-45f5-93b1-46a0344bc9f1 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:34.521240 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-59fe9561-de6d-462e-8803-5bd1b96d9c12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03624d67fc254b0c98fa42e28969d8f1'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:34.536617 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-59fe9561-de6d-462e-8803-5bd1b96d9c12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03624d67fc254b0c98fa42e28969d8f1'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:34.537474 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-59fe9561-de6d-462e-8803-5bd1b96d9c12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03624d67fc254b0c98fa42e28969d8f1'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:34.547741 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-f852c909-e63f-45f5-93b1-46a0344bc9f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '938cfff1a00b4bd198fdbc01ecfa290c', 'request_id': 'req-f852c909-e63f-45f5-93b1-46a0344bc9f1'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:34.548297 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-59fe9561-de6d-462e-8803-5bd1b96d9c12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03624d67fc254b0c98fa42e28969d8f1'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:34.549072 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-59fe9561-de6d-462e-8803-5bd1b96d9c12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03624d67fc254b0c98fa42e28969d8f1'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:34.555008 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-f852c909-e63f-45f5-93b1-46a0344bc9f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '938cfff1a00b4bd198fdbc01ecfa290c', 'request_id': 'req-f852c909-e63f-45f5-93b1-46a0344bc9f1'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:34.567837 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-f852c909-e63f-45f5-93b1-46a0344bc9f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '938cfff1a00b4bd198fdbc01ecfa290c', 'request_id': 'req-f852c909-e63f-45f5-93b1-46a0344bc9f1'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:34.570165 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1314/2614] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:21:34 2022] DELETE /identity/v3/users/03624d67fc254b0c98fa42e28969d8f1 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:21:34.575061 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2754cc3-d9a1-4937-9e5a-7cdddcc2daf8 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:34.575355 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2754cc3-d9a1-4937-9e5a-7cdddcc2daf8 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:34.575703 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2754cc3-d9a1-4937-9e5a-7cdddcc2daf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:34.582799 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-f852c909-e63f-45f5-93b1-46a0344bc9f1 admin admin] The token cache is being invalidate because project 938cfff1a00b4bd198fdbc01ecfa290c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:21:34.583116 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-f852c909-e63f-45f5-93b1-46a0344bc9f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:34.583938 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-f852c909-e63f-45f5-93b1-46a0344bc9f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:34.586287 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1301/2615] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:21:34 2022] DELETE /identity/v3/projects/938cfff1a00b4bd198fdbc01ecfa290c => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 12:21:34.590434 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f852c909-e63f-45f5-93b1-46a0344bc9f1 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:34.591778 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-f852c909-e63f-45f5-93b1-46a0344bc9f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:34.595724 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-f2754cc3-d9a1-4937-9e5a-7cdddcc2daf8 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:21:34.596966 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-f2754cc3-d9a1-4937-9e5a-7cdddcc2daf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:34.616760 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:21:34.644946 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:21:34.660696 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1315/2616] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:21:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:21:34.665311 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f852c909-e63f-45f5-93b1-46a0344bc9f1 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:34.670188 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9699d38-9fa2-4e61-9185-292cad1a0c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:34.670357 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2754cc3-d9a1-4937-9e5a-7cdddcc2daf8 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:34.671089 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9699d38-9fa2-4e61-9185-292cad1a0c8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:34.671397 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9699d38-9fa2-4e61-9185-292cad1a0c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:34.671668 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9699d38-9fa2-4e61-9185-292cad1a0c8f admin admin] PATH_INFO: `/v3/users/47557612ae6141e8a22040334f9159cf` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:34.671823 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-f2754cc3-d9a1-4937-9e5a-7cdddcc2daf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:34.672795 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9699d38-9fa2-4e61-9185-292cad1a0c8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=47557612ae6141e8a22040334f9159cf)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:34.674864 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9699d38-9fa2-4e61-9185-292cad1a0c8f admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:34.724313 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-d9699d38-9fa2-4e61-9185-292cad1a0c8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47557612ae6141e8a22040334f9159cf'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:34.731791 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2754cc3-d9a1-4937-9e5a-7cdddcc2daf8 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:34.733614 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-d9699d38-9fa2-4e61-9185-292cad1a0c8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47557612ae6141e8a22040334f9159cf'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:34.736663 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78b6287-624e-41e8-bd05-9023d7d06c82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:34.737266 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78b6287-624e-41e8-bd05-9023d7d06c82 service neutron] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:34.737542 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78b6287-624e-41e8-bd05-9023d7d06c82 service neutron] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:34.737809 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78b6287-624e-41e8-bd05-9023d7d06c82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:34.748183 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78b6287-624e-41e8-bd05-9023d7d06c82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:34.749847 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-d9699d38-9fa2-4e61-9185-292cad1a0c8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47557612ae6141e8a22040334f9159cf'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:34.750456 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78b6287-624e-41e8-bd05-9023d7d06c82 service neutron] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:34.750700 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-d9699d38-9fa2-4e61-9185-292cad1a0c8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47557612ae6141e8a22040334f9159cf'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:34.761239 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1316/2617] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:21:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 12:21:34.765718 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-d9699d38-9fa2-4e61-9185-292cad1a0c8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47557612ae6141e8a22040334f9159cf'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:34.771875 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1302/2618] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:21:34 2022] DELETE /identity/v3/users/47557612ae6141e8a22040334f9159cf => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 12:21:35.063639 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78b6287-624e-41e8-bd05-9023d7d06c82 service neutron] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:35.065684 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9699d38-9fa2-4e61-9185-292cad1a0c8f admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:35.067648 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-d9699d38-9fa2-4e61-9185-292cad1a0c8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:35.069485 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-f78b6287-624e-41e8-bd05-9023d7d06c82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:35.085727 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78b6287-624e-41e8-bd05-9023d7d06c82 service neutron] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:35.147486 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38fc48e-103b-4c8a-91a4-1a0c2db419c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:35.148039 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38fc48e-103b-4c8a-91a4-1a0c2db419c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:35.148314 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38fc48e-103b-4c8a-91a4-1a0c2db419c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:35.148489 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9699d38-9fa2-4e61-9185-292cad1a0c8f admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:35.148602 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38fc48e-103b-4c8a-91a4-1a0c2db419c5 admin admin] PATH_INFO: `/v3/projects/a895db96a8a14bb1b5e058700f5d3915` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:35.149700 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38fc48e-103b-4c8a-91a4-1a0c2db419c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a895db96a8a14bb1b5e058700f5d3915)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:35.151620 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38fc48e-103b-4c8a-91a4-1a0c2db419c5 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:35.153103 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f68b7f-00c1-46de-9313-368eaa1038bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:35.153762 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f68b7f-00c1-46de-9313-368eaa1038bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:35.154051 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f68b7f-00c1-46de-9313-368eaa1038bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:35.154320 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f68b7f-00c1-46de-9313-368eaa1038bd admin admin] PATH_INFO: `/v3/projects/0746f1408db04caf938bb2b7879e57c3` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:35.155771 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f68b7f-00c1-46de-9313-368eaa1038bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=0746f1408db04caf938bb2b7879e57c3)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:35.157655 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f68b7f-00c1-46de-9313-368eaa1038bd admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:35.203723 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-c38fc48e-103b-4c8a-91a4-1a0c2db419c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a895db96a8a14bb1b5e058700f5d3915', 'request_id': 'req-c38fc48e-103b-4c8a-91a4-1a0c2db419c5'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:35.209298 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-f6f68b7f-00c1-46de-9313-368eaa1038bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0746f1408db04caf938bb2b7879e57c3', 'request_id': 'req-f6f68b7f-00c1-46de-9313-368eaa1038bd'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:35.210947 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-c38fc48e-103b-4c8a-91a4-1a0c2db419c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a895db96a8a14bb1b5e058700f5d3915', 'request_id': 'req-c38fc48e-103b-4c8a-91a4-1a0c2db419c5'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:35.215589 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-f6f68b7f-00c1-46de-9313-368eaa1038bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0746f1408db04caf938bb2b7879e57c3', 'request_id': 'req-f6f68b7f-00c1-46de-9313-368eaa1038bd'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:35.219827 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-c38fc48e-103b-4c8a-91a4-1a0c2db419c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a895db96a8a14bb1b5e058700f5d3915', 'request_id': 'req-c38fc48e-103b-4c8a-91a4-1a0c2db419c5'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:35.224453 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-f6f68b7f-00c1-46de-9313-368eaa1038bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0746f1408db04caf938bb2b7879e57c3', 'request_id': 'req-f6f68b7f-00c1-46de-9313-368eaa1038bd'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:35.228674 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:21:35.229973 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:21:35.235594 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-c38fc48e-103b-4c8a-91a4-1a0c2db419c5 admin admin] The token cache is being invalidate because project a895db96a8a14bb1b5e058700f5d3915 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81712) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:21:35.235896 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-c38fc48e-103b-4c8a-91a4-1a0c2db419c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:35.237141 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-c38fc48e-103b-4c8a-91a4-1a0c2db419c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:35.239833 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1317/2619] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:21:35 2022] DELETE /identity/v3/projects/a895db96a8a14bb1b5e058700f5d3915 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 12:21:35.245021 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-f6f68b7f-00c1-46de-9313-368eaa1038bd admin admin] The token cache is being invalidate because project 0746f1408db04caf938bb2b7879e57c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:21:35.245021 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-f6f68b7f-00c1-46de-9313-368eaa1038bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:35.245021 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-f6f68b7f-00c1-46de-9313-368eaa1038bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:35.247411 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1303/2620] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:21:35 2022] DELETE /identity/v3/projects/0746f1408db04caf938bb2b7879e57c3 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:21:35.251751 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3763b7b5-76ce-486d-8e0d-e1582eb8fc50 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:35.251751 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3763b7b5-76ce-486d-8e0d-e1582eb8fc50 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:35.251751 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3763b7b5-76ce-486d-8e0d-e1582eb8fc50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:35.253233 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca87a6-c3d0-4679-b5cd-95ce52e7ede3 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:35.253518 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca87a6-c3d0-4679-b5cd-95ce52e7ede3 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:35.253790 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca87a6-c3d0-4679-b5cd-95ce52e7ede3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:35.269381 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-3763b7b5-76ce-486d-8e0d-e1582eb8fc50 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:21:35.270589 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-3763b7b5-76ce-486d-8e0d-e1582eb8fc50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:35.277534 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-66ca87a6-c3d0-4679-b5cd-95ce52e7ede3 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:21:35.278835 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-66ca87a6-c3d0-4679-b5cd-95ce52e7ede3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:35.337174 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1318/2621] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:21:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:21:35.345285 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87c1c3-2c25-467c-bbc8-7a70d103fb62 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:35.345874 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87c1c3-2c25-467c-bbc8-7a70d103fb62 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:35.345874 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87c1c3-2c25-467c-bbc8-7a70d103fb62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:35.359365 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1304/2622] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:21:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:21:35.364273 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452fc44b-eaf7-4096-ab24-e7b52dc2d309 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:35.364552 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452fc44b-eaf7-4096-ab24-e7b52dc2d309 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:35.364821 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452fc44b-eaf7-4096-ab24-e7b52dc2d309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:35.366528 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-ca87c1c3-2c25-467c-bbc8-7a70d103fb62 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:21:35.367803 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-ca87c1c3-2c25-467c-bbc8-7a70d103fb62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:35.379304 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1319/2623] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:21:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:21:35.383289 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca87c1c3-2c25-467c-bbc8-7a70d103fb62 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:35.389883 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-452fc44b-eaf7-4096-ab24-e7b52dc2d309 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:21:35.391137 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-452fc44b-eaf7-4096-ab24-e7b52dc2d309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:35.392418 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca87c1c3-2c25-467c-bbc8-7a70d103fb62 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:35.397056 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb7fdb1-bdb4-4f6a-abea-29de15a65173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:35.397764 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb7fdb1-bdb4-4f6a-abea-29de15a65173 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:35.398061 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb7fdb1-bdb4-4f6a-abea-29de15a65173 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:35.398419 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb7fdb1-bdb4-4f6a-abea-29de15a65173 admin admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:35.399127 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb7fdb1-bdb4-4f6a-abea-29de15a65173 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:21:35.399552 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1305/2624] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:21:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:21:35.399677 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb7fdb1-bdb4-4f6a-abea-29de15a65173 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:35.401220 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb7fdb1-bdb4-4f6a-abea-29de15a65173 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:35.403423 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452fc44b-eaf7-4096-ab24-e7b52dc2d309 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:35.413644 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452fc44b-eaf7-4096-ab24-e7b52dc2d309 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:35.414549 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1320/2625] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:21:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:21:35.418326 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb7fdb1-bdb4-4f6a-abea-29de15a65173 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:35.418462 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18f643c-3501-403b-be9b-6ea93fe02a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:35.418990 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18f643c-3501-403b-be9b-6ea93fe02a5c admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:35.419300 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18f643c-3501-403b-be9b-6ea93fe02a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:35.419575 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18f643c-3501-403b-be9b-6ea93fe02a5c admin admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:35.420211 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18f643c-3501-403b-be9b-6ea93fe02a5c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:21:35.421086 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18f643c-3501-403b-be9b-6ea93fe02a5c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:35.422665 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18f643c-3501-403b-be9b-6ea93fe02a5c admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:35.428717 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb7fdb1-bdb4-4f6a-abea-29de15a65173 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:35.433333 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3ea238-f433-418f-b191-80d3a378b6cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:35.433896 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ea238-f433-418f-b191-80d3a378b6cf admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:35.434172 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ea238-f433-418f-b191-80d3a378b6cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:35.434442 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ea238-f433-418f-b191-80d3a378b6cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:35.438453 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1306/2626] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:21:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:21:35.438602 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3ea238-f433-418f-b191-80d3a378b6cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:35.438602 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3ea238-f433-418f-b191-80d3a378b6cf admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:35.439251 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18f643c-3501-403b-be9b-6ea93fe02a5c admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:35.448677 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18f643c-3501-403b-be9b-6ea93fe02a5c admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:35.453314 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2821535-8b2b-4290-bebf-c40d8a46f277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:35.453891 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2821535-8b2b-4290-bebf-c40d8a46f277 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:35.454176 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2821535-8b2b-4290-bebf-c40d8a46f277 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:35.454449 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2821535-8b2b-4290-bebf-c40d8a46f277 admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:35.455067 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2821535-8b2b-4290-bebf-c40d8a46f277 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:35.457173 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2821535-8b2b-4290-bebf-c40d8a46f277 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:35.468911 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1321/2627] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:21:35 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:21:35.469926 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3ea238-f433-418f-b191-80d3a378b6cf admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:35.486698 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1307/2628] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:21:35 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 12:21:35.491062 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2821535-8b2b-4290-bebf-c40d8a46f277 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:35.533747 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3ea238-f433-418f-b191-80d3a378b6cf admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:35.552319 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2821535-8b2b-4290-bebf-c40d8a46f277 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:35.557314 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ac5af6-9d33-43fa-b82e-67ab4c586723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:35.557981 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac5af6-9d33-43fa-b82e-67ab4c586723 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:35.558241 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac5af6-9d33-43fa-b82e-67ab4c586723 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:35.558507 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac5af6-9d33-43fa-b82e-67ab4c586723 admin admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:35.559116 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ac5af6-9d33-43fa-b82e-67ab4c586723 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:35.561079 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ac5af6-9d33-43fa-b82e-67ab4c586723 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:35.583256 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1308/2629] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:21:35 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 12:21:35.587247 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ac5af6-9d33-43fa-b82e-67ab4c586723 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:35.593101 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9ce76d-0b67-47eb-b795-f1d806021740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:35.593720 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ce76d-0b67-47eb-b795-f1d806021740 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:35.594001 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ce76d-0b67-47eb-b795-f1d806021740 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:35.594281 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ce76d-0b67-47eb-b795-f1d806021740 admin admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:35.594906 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9ce76d-0b67-47eb-b795-f1d806021740 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:35.596914 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9ce76d-0b67-47eb-b795-f1d806021740 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:35.597400 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ac5af6-9d33-43fa-b82e-67ab4c586723 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:35.601952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f282166-fb80-4407-ad44-15a9586f5928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:35.602533 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f282166-fb80-4407-ad44-15a9586f5928 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:35.602811 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f282166-fb80-4407-ad44-15a9586f5928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:35.603085 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f282166-fb80-4407-ad44-15a9586f5928 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:35.603752 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f282166-fb80-4407-ad44-15a9586f5928 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:35.605324 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f282166-fb80-4407-ad44-15a9586f5928 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:35.606828 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:21:35.617223 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1309/2630] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:21:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 12:21:35.620732 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1322/2631] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:21:35 2022] POST /identity/v3/users => generated 286 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:21:35.621068 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f282166-fb80-4407-ad44-15a9586f5928 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:35.624639 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9ce76d-0b67-47eb-b795-f1d806021740 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:35.630838 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f282166-fb80-4407-ad44-15a9586f5928 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:35.634416 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9ce76d-0b67-47eb-b795-f1d806021740 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:35.635790 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114a333b-7e2c-48b8-9a74-c2769c147f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:35.636377 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a333b-7e2c-48b8-9a74-c2769c147f52 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:35.636657 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a333b-7e2c-48b8-9a74-c2769c147f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:35.636934 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a333b-7e2c-48b8-9a74-c2769c147f52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:35.637690 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114a333b-7e2c-48b8-9a74-c2769c147f52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:35.639057 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091bb53e-c392-417b-a666-a4f82c0edcc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:35.639347 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114a333b-7e2c-48b8-9a74-c2769c147f52 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:35.639697 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091bb53e-c392-417b-a666-a4f82c0edcc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:35.639983 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091bb53e-c392-417b-a666-a4f82c0edcc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:35.640259 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091bb53e-c392-417b-a666-a4f82c0edcc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:35.640907 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091bb53e-c392-417b-a666-a4f82c0edcc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:35.642504 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091bb53e-c392-417b-a666-a4f82c0edcc4 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:35.651495 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1310/2632] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:21:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 12:21:35.652902 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1323/2633] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:21:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:21:35.656218 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114a333b-7e2c-48b8-9a74-c2769c147f52 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:35.657111 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091bb53e-c392-417b-a666-a4f82c0edcc4 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:35.665859 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114a333b-7e2c-48b8-9a74-c2769c147f52 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:35.667403 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091bb53e-c392-417b-a666-a4f82c0edcc4 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:35.670390 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead041ab-3034-4cc2-b569-3785b793f263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:35.671624 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead041ab-3034-4cc2-b569-3785b793f263 admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:35.671901 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead041ab-3034-4cc2-b569-3785b793f263 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:35.672171 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead041ab-3034-4cc2-b569-3785b793f263 admin admin] PATH_INFO: `/v3/projects/30bf9bfe9e194ecbbfe5f16bfdf247d2/users/fef58a48b3e54284b0715d6fb23dd834/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:35.672817 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c24b8c-feb6-4ebe-9971-773a952de914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:35.673399 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c24b8c-feb6-4ebe-9971-773a952de914 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:35.673683 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c24b8c-feb6-4ebe-9971-773a952de914 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:35.673960 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c24b8c-feb6-4ebe-9971-773a952de914 admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:35.674588 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c24b8c-feb6-4ebe-9971-773a952de914 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:35.676197 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c24b8c-feb6-4ebe-9971-773a952de914 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:35.686445 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead041ab-3034-4cc2-b569-3785b793f263 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30bf9bfe9e194ecbbfe5f16bfdf247d2, user_id=fef58a48b3e54284b0715d6fb23dd834, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:35.687043 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1324/2634] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:21:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:21:35.690525 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead041ab-3034-4cc2-b569-3785b793f263 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:35.691157 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c24b8c-feb6-4ebe-9971-773a952de914 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:35.692354 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:DELETE=1,keystone:SELECT=94,keystone:INSERT=5 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:21:35.703553 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c24b8c-feb6-4ebe-9971-773a952de914 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:35.705015 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1311/2635] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:21:35 2022] PUT /identity/v3/projects/30bf9bfe9e194ecbbfe5f16bfdf247d2/users/fef58a48b3e54284b0715d6fb23dd834/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:21:35.716015 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1936808f-0a89-41e4-b968-2f234f1508f6 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:35.716300 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1936808f-0a89-41e4-b968-2f234f1508f6 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:35.716578 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1936808f-0a89-41e4-b968-2f234f1508f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:35.751635 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-1936808f-0a89-41e4-b968-2f234f1508f6 None None] MFA Rules not processed for user `fef58a48b3e54284b0715d6fb23dd834`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:21:35.752855 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-1936808f-0a89-41e4-b968-2f234f1508f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:35.768270 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d65a66-5083-4a61-92a8-d11d11610627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:35.768928 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d65a66-5083-4a61-92a8-d11d11610627 admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:35.769211 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d65a66-5083-4a61-92a8-d11d11610627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:35.769484 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d65a66-5083-4a61-92a8-d11d11610627 admin admin] PATH_INFO: `/v3/projects/5f4b94b7c6e24398806841d213a180a3/users/de38de38b9474f66adcb3b8a38566e3b/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:35.784379 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d65a66-5083-4a61-92a8-d11d11610627 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f4b94b7c6e24398806841d213a180a3, user_id=de38de38b9474f66adcb3b8a38566e3b, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:35.787035 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d65a66-5083-4a61-92a8-d11d11610627 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:35.800187 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1325/2636] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:21:35 2022] PUT /identity/v3/projects/5f4b94b7c6e24398806841d213a180a3/users/de38de38b9474f66adcb3b8a38566e3b/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 12:21:35.812139 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e399713f-5825-4388-9d5a-a679f40461fd None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:35.812541 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e399713f-5825-4388-9d5a-a679f40461fd None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:35.812911 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e399713f-5825-4388-9d5a-a679f40461fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:35.818707 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1312/2637] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:21:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:21:35.830488 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1936808f-0a89-41e4-b968-2f234f1508f6 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:35.831924 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-1936808f-0a89-41e4-b968-2f234f1508f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:35.844008 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-e399713f-5825-4388-9d5a-a679f40461fd None None] MFA Rules not processed for user `de38de38b9474f66adcb3b8a38566e3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:21:35.845210 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-e399713f-5825-4388-9d5a-a679f40461fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:35.891969 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1936808f-0a89-41e4-b968-2f234f1508f6 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:35.911387 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1326/2638] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:21:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:21:35.922879 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e399713f-5825-4388-9d5a-a679f40461fd None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:35.924276 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-e399713f-5825-4388-9d5a-a679f40461fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:35.941754 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e47f65-b03c-46a4-9082-49afa97fb64d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:35.942360 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e47f65-b03c-46a4-9082-49afa97fb64d service cinder] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:35.942646 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e47f65-b03c-46a4-9082-49afa97fb64d service cinder] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:35.942919 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e47f65-b03c-46a4-9082-49afa97fb64d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:35.971427 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:21:35.983590 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e399713f-5825-4388-9d5a-a679f40461fd None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:35.987333 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc0817d-2b52-41d2-a674-a901442bd767 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:35.988191 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc0817d-2b52-41d2-a674-a901442bd767 service cinder] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:35.988474 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc0817d-2b52-41d2-a674-a901442bd767 service cinder] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:35.988744 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc0817d-2b52-41d2-a674-a901442bd767 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:35.997215 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc0817d-2b52-41d2-a674-a901442bd767 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:35.998781 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc0817d-2b52-41d2-a674-a901442bd767 service cinder] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:35.999149 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:21:36.001797 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e47f65-b03c-46a4-9082-49afa97fb64d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:36.003577 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e47f65-b03c-46a4-9082-49afa97fb64d service cinder] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:36.007983 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1327/2639] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:21:35 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 12:21:36.012614 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1313/2640] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:21:35 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 12:21:36.030131 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc0817d-2b52-41d2-a674-a901442bd767 service cinder] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:36.094553 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc0817d-2b52-41d2-a674-a901442bd767 service cinder] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:36.099444 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b1723e-541e-48f8-a719-b5fa33eaf361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:36.100075 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b1723e-541e-48f8-a719-b5fa33eaf361 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:36.100540 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b1723e-541e-48f8-a719-b5fa33eaf361 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:36.100819 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b1723e-541e-48f8-a719-b5fa33eaf361 admin admin] PATH_INFO: `/v3/users/de38de38b9474f66adcb3b8a38566e3b` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:36.101981 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b1723e-541e-48f8-a719-b5fa33eaf361 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de38de38b9474f66adcb3b8a38566e3b)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:36.103974 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b1723e-541e-48f8-a719-b5fa33eaf361 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:36.154488 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-80b1723e-541e-48f8-a719-b5fa33eaf361 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de38de38b9474f66adcb3b8a38566e3b'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:36.164174 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-80b1723e-541e-48f8-a719-b5fa33eaf361 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de38de38b9474f66adcb3b8a38566e3b'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:36.164831 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-80b1723e-541e-48f8-a719-b5fa33eaf361 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de38de38b9474f66adcb3b8a38566e3b'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:36.172909 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-80b1723e-541e-48f8-a719-b5fa33eaf361 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de38de38b9474f66adcb3b8a38566e3b'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:36.173625 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-80b1723e-541e-48f8-a719-b5fa33eaf361 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de38de38b9474f66adcb3b8a38566e3b'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:36.192226 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1328/2641] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:21:36 2022] DELETE /identity/v3/users/de38de38b9474f66adcb3b8a38566e3b => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:21:36.196743 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2370a560-9f8b-4021-b862-b396463cafb2 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:36.197038 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2370a560-9f8b-4021-b862-b396463cafb2 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:36.197317 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2370a560-9f8b-4021-b862-b396463cafb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:36.216375 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-2370a560-9f8b-4021-b862-b396463cafb2 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:21:36.218301 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-2370a560-9f8b-4021-b862-b396463cafb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:36.306220 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1314/2642] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:21:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:21:36.316590 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b1723e-541e-48f8-a719-b5fa33eaf361 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:36.318172 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-80b1723e-541e-48f8-a719-b5fa33eaf361 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:36.378290 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b1723e-541e-48f8-a719-b5fa33eaf361 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:36.381950 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d09dc1-e2db-4a9a-a6a4-5566dfe8ae5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:36.382747 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d09dc1-e2db-4a9a-a6a4-5566dfe8ae5d service neutron] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:36.383036 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d09dc1-e2db-4a9a-a6a4-5566dfe8ae5d service neutron] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:36.383340 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d09dc1-e2db-4a9a-a6a4-5566dfe8ae5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:36.393101 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d09dc1-e2db-4a9a-a6a4-5566dfe8ae5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:36.394556 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d09dc1-e2db-4a9a-a6a4-5566dfe8ae5d service neutron] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:36.404584 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1329/2643] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:21:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:21:36.684739 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2370a560-9f8b-4021-b862-b396463cafb2 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:36.686629 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-2370a560-9f8b-4021-b862-b396463cafb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:36.700222 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2370a560-9f8b-4021-b862-b396463cafb2 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:36.705881 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80589511-23fe-4163-a4a6-aca602df72e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:36.706619 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80589511-23fe-4163-a4a6-aca602df72e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:36.706969 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80589511-23fe-4163-a4a6-aca602df72e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:36.707888 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80589511-23fe-4163-a4a6-aca602df72e7 admin admin] PATH_INFO: `/v3/projects/5f4b94b7c6e24398806841d213a180a3` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:36.709322 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80589511-23fe-4163-a4a6-aca602df72e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f4b94b7c6e24398806841d213a180a3)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:36.711300 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80589511-23fe-4163-a4a6-aca602df72e7 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:36.763279 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-80589511-23fe-4163-a4a6-aca602df72e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f4b94b7c6e24398806841d213a180a3', 'request_id': 'req-80589511-23fe-4163-a4a6-aca602df72e7'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:36.769973 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-80589511-23fe-4163-a4a6-aca602df72e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f4b94b7c6e24398806841d213a180a3', 'request_id': 'req-80589511-23fe-4163-a4a6-aca602df72e7'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:36.777284 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-80589511-23fe-4163-a4a6-aca602df72e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f4b94b7c6e24398806841d213a180a3', 'request_id': 'req-80589511-23fe-4163-a4a6-aca602df72e7'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:36.788239 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-80589511-23fe-4163-a4a6-aca602df72e7 admin admin] The token cache is being invalidate because project 5f4b94b7c6e24398806841d213a180a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:21:36.788531 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-80589511-23fe-4163-a4a6-aca602df72e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:36.789148 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-80589511-23fe-4163-a4a6-aca602df72e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:36.790645 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1315/2644] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:21:36 2022] DELETE /identity/v3/projects/5f4b94b7c6e24398806841d213a180a3 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:21:36.795644 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be412a-6ef3-473a-8fd3-61e9a552e089 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:36.795926 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be412a-6ef3-473a-8fd3-61e9a552e089 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:36.796225 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be412a-6ef3-473a-8fd3-61e9a552e089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:36.815490 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-90be412a-6ef3-473a-8fd3-61e9a552e089 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:21:36.816663 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-90be412a-6ef3-473a-8fd3-61e9a552e089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:36.820699 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:21:36.877127 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1330/2645] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:21:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:21:36.884763 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9770e8ed-feec-4171-8820-021bb3c9bc9a None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:36.885256 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9770e8ed-feec-4171-8820-021bb3c9bc9a None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:36.885369 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9770e8ed-feec-4171-8820-021bb3c9bc9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:36.904905 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-9770e8ed-feec-4171-8820-021bb3c9bc9a None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:21:36.906082 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-9770e8ed-feec-4171-8820-021bb3c9bc9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:36.913791 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1316/2646] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:21:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:21:36.917814 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90be412a-6ef3-473a-8fd3-61e9a552e089 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:36.927263 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90be412a-6ef3-473a-8fd3-61e9a552e089 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:36.931757 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e0eaf5-475b-4045-a350-5c652c2e672a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:36.932280 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e0eaf5-475b-4045-a350-5c652c2e672a admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:36.932555 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e0eaf5-475b-4045-a350-5c652c2e672a admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:36.932822 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e0eaf5-475b-4045-a350-5c652c2e672a admin admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:36.933452 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e0eaf5-475b-4045-a350-5c652c2e672a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:21:36.933749 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e0eaf5-475b-4045-a350-5c652c2e672a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:36.935377 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e0eaf5-475b-4045-a350-5c652c2e672a admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:36.947100 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1331/2647] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:21:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:21:36.951146 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9770e8ed-feec-4171-8820-021bb3c9bc9a None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:36.961205 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9770e8ed-feec-4171-8820-021bb3c9bc9a None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:36.965859 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f5cc91-5d63-4705-afa4-ede62321773b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:36.966453 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5cc91-5d63-4705-afa4-ede62321773b admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:36.966734 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5cc91-5d63-4705-afa4-ede62321773b admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:36.967025 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5cc91-5d63-4705-afa4-ede62321773b admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:36.967698 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f5cc91-5d63-4705-afa4-ede62321773b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:36.969586 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f5cc91-5d63-4705-afa4-ede62321773b admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:36.996727 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1317/2648] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:21:36 2022] POST /identity/v3/projects => generated 359 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:21:37.001355 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e0eaf5-475b-4045-a350-5c652c2e672a admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:37.011262 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e0eaf5-475b-4045-a350-5c652c2e672a admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:37.067580 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9e7be7-d067-4943-bee2-36128332ab52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:37.068216 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e7be7-d067-4943-bee2-36128332ab52 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:37.068496 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e7be7-d067-4943-bee2-36128332ab52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:37.068770 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e7be7-d067-4943-bee2-36128332ab52 admin admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:37.069391 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9e7be7-d067-4943-bee2-36128332ab52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:37.071329 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9e7be7-d067-4943-bee2-36128332ab52 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:37.094097 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1332/2649] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:21:37 2022] POST /identity/v3/users => generated 280 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:21:37.098188 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f5cc91-5d63-4705-afa4-ede62321773b admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:37.108364 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f5cc91-5d63-4705-afa4-ede62321773b admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:37.113348 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e966cdda-a48c-4ee1-96f6-97527361686e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:37.113941 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e966cdda-a48c-4ee1-96f6-97527361686e admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:37.114219 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e966cdda-a48c-4ee1-96f6-97527361686e admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:37.114489 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e966cdda-a48c-4ee1-96f6-97527361686e admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:37.115126 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e966cdda-a48c-4ee1-96f6-97527361686e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:37.116762 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e966cdda-a48c-4ee1-96f6-97527361686e admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:37.127046 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1318/2650] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:21:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:21:37.130686 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9e7be7-d067-4943-bee2-36128332ab52 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:37.140750 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9e7be7-d067-4943-bee2-36128332ab52 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:37.145904 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ace6dbc-715d-4bc3-acd0-cd1b29c04ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:37.146492 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ace6dbc-715d-4bc3-acd0-cd1b29c04ed0 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:37.146770 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ace6dbc-715d-4bc3-acd0-cd1b29c04ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:37.147043 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ace6dbc-715d-4bc3-acd0-cd1b29c04ed0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:37.147792 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ace6dbc-715d-4bc3-acd0-cd1b29c04ed0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:37.149502 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ace6dbc-715d-4bc3-acd0-cd1b29c04ed0 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:37.160233 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1333/2651] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:21:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:21:37.164394 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e966cdda-a48c-4ee1-96f6-97527361686e admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:37.175309 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e966cdda-a48c-4ee1-96f6-97527361686e admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:37.179965 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e1c317-f882-4c5b-8ce8-8e4732a8d124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:37.180606 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e1c317-f882-4c5b-8ce8-8e4732a8d124 admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:37.180886 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e1c317-f882-4c5b-8ce8-8e4732a8d124 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:37.181158 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e1c317-f882-4c5b-8ce8-8e4732a8d124 admin admin] PATH_INFO: `/v3/projects/cb96086ed15d4be1a226f24c942eac95/users/34f75b2cbce341a2a890a7abc6ea4682/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:37.197053 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e1c317-f882-4c5b-8ce8-8e4732a8d124 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb96086ed15d4be1a226f24c942eac95, user_id=34f75b2cbce341a2a890a7abc6ea4682, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:37.199648 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e1c317-f882-4c5b-8ce8-8e4732a8d124 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:37.212306 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1319/2652] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:21:37 2022] PUT /identity/v3/projects/cb96086ed15d4be1a226f24c942eac95/users/34f75b2cbce341a2a890a7abc6ea4682/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:21:37.222549 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f4f4ec-b766-4645-a351-cebd1236b7ea None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:37.222869 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f4f4ec-b766-4645-a351-cebd1236b7ea None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:37.223167 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f4f4ec-b766-4645-a351-cebd1236b7ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:37.251036 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-b2f4f4ec-b766-4645-a351-cebd1236b7ea None None] MFA Rules not processed for user `34f75b2cbce341a2a890a7abc6ea4682`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:21:37.252441 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-b2f4f4ec-b766-4645-a351-cebd1236b7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:37.315569 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1334/2653] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:21:37 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:21:37.348056 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e1c317-f882-4c5b-8ce8-8e4732a8d124 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:37.349811 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-98e1c317-f882-4c5b-8ce8-8e4732a8d124 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:37.369019 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:21:37.409536 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e1c317-f882-4c5b-8ce8-8e4732a8d124 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:37.413174 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6444c910-b3e6-4467-a1be-f3f72ab1aec0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:37.413770 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444c910-b3e6-4467-a1be-f3f72ab1aec0 service cinder] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:37.414051 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444c910-b3e6-4467-a1be-f3f72ab1aec0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:37.414319 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444c910-b3e6-4467-a1be-f3f72ab1aec0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:37.422981 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6444c910-b3e6-4467-a1be-f3f72ab1aec0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:37.424422 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6444c910-b3e6-4467-a1be-f3f72ab1aec0 service cinder] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:37.433097 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1320/2654] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:21:37 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:21:37.449439 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f4f4ec-b766-4645-a351-cebd1236b7ea None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:37.455561 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:21:37.520601 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f4f4ec-b766-4645-a351-cebd1236b7ea None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:37.526338 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4e6dbd-2f91-433f-9f89-61ea81ce77cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:37.527042 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e6dbd-2f91-433f-9f89-61ea81ce77cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:37.527418 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e6dbd-2f91-433f-9f89-61ea81ce77cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:37.527762 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e6dbd-2f91-433f-9f89-61ea81ce77cb admin admin] PATH_INFO: `/v3/users/34f75b2cbce341a2a890a7abc6ea4682` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:37.528960 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4e6dbd-2f91-433f-9f89-61ea81ce77cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=34f75b2cbce341a2a890a7abc6ea4682)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:37.531851 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4e6dbd-2f91-433f-9f89-61ea81ce77cb admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:37.711946 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-cb4e6dbd-2f91-433f-9f89-61ea81ce77cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34f75b2cbce341a2a890a7abc6ea4682'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:37.720038 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-cb4e6dbd-2f91-433f-9f89-61ea81ce77cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34f75b2cbce341a2a890a7abc6ea4682'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:37.720725 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-cb4e6dbd-2f91-433f-9f89-61ea81ce77cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34f75b2cbce341a2a890a7abc6ea4682'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:37.728024 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-cb4e6dbd-2f91-433f-9f89-61ea81ce77cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34f75b2cbce341a2a890a7abc6ea4682'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:37.728816 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-cb4e6dbd-2f91-433f-9f89-61ea81ce77cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34f75b2cbce341a2a890a7abc6ea4682'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:37.754591 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1335/2655] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:21:37 2022] DELETE /identity/v3/users/34f75b2cbce341a2a890a7abc6ea4682 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:21:37.759453 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a426c7a5-6122-4b30-8b92-f34894dfb1fb None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:37.759781 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a426c7a5-6122-4b30-8b92-f34894dfb1fb None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:37.760091 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a426c7a5-6122-4b30-8b92-f34894dfb1fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:37.778204 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-a426c7a5-6122-4b30-8b92-f34894dfb1fb None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:21:37.779405 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-a426c7a5-6122-4b30-8b92-f34894dfb1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:37.837131 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1321/2656] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:21:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:21:37.847685 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4e6dbd-2f91-433f-9f89-61ea81ce77cb admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:37.849076 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-cb4e6dbd-2f91-433f-9f89-61ea81ce77cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:37.906513 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4e6dbd-2f91-433f-9f89-61ea81ce77cb admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:37.910109 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d5cd3f-4c53-4836-909d-06d64a4636f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:37.910707 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5cd3f-4c53-4836-909d-06d64a4636f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:37.910983 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5cd3f-4c53-4836-909d-06d64a4636f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:37.911281 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d5cd3f-4c53-4836-909d-06d64a4636f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:37.920099 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d5cd3f-4c53-4836-909d-06d64a4636f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:37.922247 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d5cd3f-4c53-4836-909d-06d64a4636f7 service neutron] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:37.931441 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1336/2657] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:21:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:21:38.192328 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a426c7a5-6122-4b30-8b92-f34894dfb1fb None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:38.193771 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-a426c7a5-6122-4b30-8b92-f34894dfb1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:38.207457 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a426c7a5-6122-4b30-8b92-f34894dfb1fb None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:38.212093 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587c4e9f-4a24-4d9a-8c9f-50cc078145e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:38.212779 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c4e9f-4a24-4d9a-8c9f-50cc078145e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:38.213703 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c4e9f-4a24-4d9a-8c9f-50cc078145e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:38.214345 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587c4e9f-4a24-4d9a-8c9f-50cc078145e6 admin admin] PATH_INFO: `/v3/projects/cb96086ed15d4be1a226f24c942eac95` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:38.215515 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587c4e9f-4a24-4d9a-8c9f-50cc078145e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb96086ed15d4be1a226f24c942eac95)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:38.217606 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587c4e9f-4a24-4d9a-8c9f-50cc078145e6 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:38.265670 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-587c4e9f-4a24-4d9a-8c9f-50cc078145e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb96086ed15d4be1a226f24c942eac95', 'request_id': 'req-587c4e9f-4a24-4d9a-8c9f-50cc078145e6'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:38.271817 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-587c4e9f-4a24-4d9a-8c9f-50cc078145e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb96086ed15d4be1a226f24c942eac95', 'request_id': 'req-587c4e9f-4a24-4d9a-8c9f-50cc078145e6'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:38.278826 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-587c4e9f-4a24-4d9a-8c9f-50cc078145e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb96086ed15d4be1a226f24c942eac95', 'request_id': 'req-587c4e9f-4a24-4d9a-8c9f-50cc078145e6'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:38.290272 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-587c4e9f-4a24-4d9a-8c9f-50cc078145e6 admin admin] The token cache is being invalidate because project cb96086ed15d4be1a226f24c942eac95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:21:38.290559 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-587c4e9f-4a24-4d9a-8c9f-50cc078145e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:38.291237 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-587c4e9f-4a24-4d9a-8c9f-50cc078145e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:38.292718 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1322/2658] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:21:38 2022] DELETE /identity/v3/projects/cb96086ed15d4be1a226f24c942eac95 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:21:38.298917 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0221c9e-137c-4109-bafc-736d8e4f6044 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:38.299250 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0221c9e-137c-4109-bafc-736d8e4f6044 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:38.299526 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0221c9e-137c-4109-bafc-736d8e4f6044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:38.318667 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-f0221c9e-137c-4109-bafc-736d8e4f6044 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:21:38.319886 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-f0221c9e-137c-4109-bafc-736d8e4f6044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:38.340734 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:21:38.381670 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1337/2659] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:21:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:21:38.391375 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab09a96-b847-4eaf-9a45-0eb1a8edb148 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:38.391669 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab09a96-b847-4eaf-9a45-0eb1a8edb148 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:38.391948 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab09a96-b847-4eaf-9a45-0eb1a8edb148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:38.411500 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-7ab09a96-b847-4eaf-9a45-0eb1a8edb148 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:21:38.412699 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-7ab09a96-b847-4eaf-9a45-0eb1a8edb148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:38.420580 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1323/2660] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:21:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:21:38.424761 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0221c9e-137c-4109-bafc-736d8e4f6044 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:38.434294 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0221c9e-137c-4109-bafc-736d8e4f6044 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:38.439481 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf73b957-7c66-484c-86be-b389145858cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:38.440067 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf73b957-7c66-484c-86be-b389145858cf admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:38.440343 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf73b957-7c66-484c-86be-b389145858cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:38.440611 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf73b957-7c66-484c-86be-b389145858cf admin admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:38.441249 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf73b957-7c66-484c-86be-b389145858cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:21:38.441547 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf73b957-7c66-484c-86be-b389145858cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:38.443122 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf73b957-7c66-484c-86be-b389145858cf admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:38.455040 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1338/2661] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:21:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:21:38.459132 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab09a96-b847-4eaf-9a45-0eb1a8edb148 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:38.468434 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab09a96-b847-4eaf-9a45-0eb1a8edb148 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:38.473001 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d14cadd-e32a-431b-ac66-70ac7f702aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:38.473586 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d14cadd-e32a-431b-ac66-70ac7f702aa7 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:38.473861 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d14cadd-e32a-431b-ac66-70ac7f702aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:38.474127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d14cadd-e32a-431b-ac66-70ac7f702aa7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:38.474737 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d14cadd-e32a-431b-ac66-70ac7f702aa7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:38.476673 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d14cadd-e32a-431b-ac66-70ac7f702aa7 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:38.504156 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1324/2662] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:21:38 2022] POST /identity/v3/projects => generated 389 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:21:38.508717 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf73b957-7c66-484c-86be-b389145858cf admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:38.519210 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf73b957-7c66-484c-86be-b389145858cf admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:38.576569 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffa0eec-244c-4878-8984-4dd6fa202d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:38.578006 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa0eec-244c-4878-8984-4dd6fa202d6b admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:38.578291 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa0eec-244c-4878-8984-4dd6fa202d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:38.578564 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa0eec-244c-4878-8984-4dd6fa202d6b admin admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:38.579294 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffa0eec-244c-4878-8984-4dd6fa202d6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:38.581283 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffa0eec-244c-4878-8984-4dd6fa202d6b admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:38.601191 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1339/2663] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:21:38 2022] POST /identity/v3/users => generated 295 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 12:21:38.605256 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d14cadd-e32a-431b-ac66-70ac7f702aa7 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:38.615526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d14cadd-e32a-431b-ac66-70ac7f702aa7 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:38.620040 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d75e57-83b4-40e1-9eb6-34e8415958fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:38.620625 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d75e57-83b4-40e1-9eb6-34e8415958fb admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:38.620901 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d75e57-83b4-40e1-9eb6-34e8415958fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:38.621167 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d75e57-83b4-40e1-9eb6-34e8415958fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:38.621795 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d75e57-83b4-40e1-9eb6-34e8415958fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:38.623394 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d75e57-83b4-40e1-9eb6-34e8415958fb admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:38.633624 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1325/2664] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:21:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 12:21:38.637299 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffa0eec-244c-4878-8984-4dd6fa202d6b admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:38.647702 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffa0eec-244c-4878-8984-4dd6fa202d6b admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:38.652427 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b442ce-ab0e-413f-9355-ec3935abc61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:38.653007 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b442ce-ab0e-413f-9355-ec3935abc61d admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:38.653282 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b442ce-ab0e-413f-9355-ec3935abc61d admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:38.653550 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b442ce-ab0e-413f-9355-ec3935abc61d admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:38.654216 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b442ce-ab0e-413f-9355-ec3935abc61d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:38.655883 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b442ce-ab0e-413f-9355-ec3935abc61d admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:38.665558 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1340/2665] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:21:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:21:38.669498 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d75e57-83b4-40e1-9eb6-34e8415958fb admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:38.675485 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:21:38.681785 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d75e57-83b4-40e1-9eb6-34e8415958fb admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:38.688006 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52086762-41d6-411a-b320-0f4747cba60a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:38.688640 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52086762-41d6-411a-b320-0f4747cba60a admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:38.688913 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52086762-41d6-411a-b320-0f4747cba60a admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:38.689183 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52086762-41d6-411a-b320-0f4747cba60a admin admin] PATH_INFO: `/v3/projects/50e8bcf90bd149589df963d40ad79f32/users/e0ac29547caa4204b7f30ebb143e0032/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:38.703354 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52086762-41d6-411a-b320-0f4747cba60a admin admin] RBAC: Authorizing `identity:create_grant(project_id=50e8bcf90bd149589df963d40ad79f32, user_id=e0ac29547caa4204b7f30ebb143e0032, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:38.705967 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52086762-41d6-411a-b320-0f4747cba60a admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:38.718176 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1326/2666] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:21:38 2022] PUT /identity/v3/projects/50e8bcf90bd149589df963d40ad79f32/users/e0ac29547caa4204b7f30ebb143e0032/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:21:38.728369 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fd4cba-4db8-4952-8b87-4ef594824c0c None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:38.728656 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fd4cba-4db8-4952-8b87-4ef594824c0c None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:38.728931 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fd4cba-4db8-4952-8b87-4ef594824c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:38.758469 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-79fd4cba-4db8-4952-8b87-4ef594824c0c None None] MFA Rules not processed for user `e0ac29547caa4204b7f30ebb143e0032`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:21:38.759707 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-79fd4cba-4db8-4952-8b87-4ef594824c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:38.823492 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1341/2667] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:21:38 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:21:38.833966 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52086762-41d6-411a-b320-0f4747cba60a admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:38.835764 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-52086762-41d6-411a-b320-0f4747cba60a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:38.894543 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52086762-41d6-411a-b320-0f4747cba60a admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:38.898235 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e34cb6e-4583-4d96-968e-fee1cc702607 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:38.898897 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e34cb6e-4583-4d96-968e-fee1cc702607 service cinder] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:38.899175 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e34cb6e-4583-4d96-968e-fee1cc702607 service cinder] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:38.899479 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e34cb6e-4583-4d96-968e-fee1cc702607 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:38.908264 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e34cb6e-4583-4d96-968e-fee1cc702607 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:38.909694 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e34cb6e-4583-4d96-968e-fee1cc702607 service cinder] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:38.918685 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1327/2668] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:21:38 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:21:45.521350 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fd4cba-4db8-4952-8b87-4ef594824c0c None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:45.522742 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-79fd4cba-4db8-4952-8b87-4ef594824c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:45.549051 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:21:45.587130 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fd4cba-4db8-4952-8b87-4ef594824c0c None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:45.591616 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a23ae4-b808-489e-aa79-b7b241d76f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:45.592764 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a23ae4-b808-489e-aa79-b7b241d76f6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:45.593044 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a23ae4-b808-489e-aa79-b7b241d76f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:45.593312 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a23ae4-b808-489e-aa79-b7b241d76f6b admin admin] PATH_INFO: `/v3/users/fef58a48b3e54284b0715d6fb23dd834` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:45.594427 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a23ae4-b808-489e-aa79-b7b241d76f6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fef58a48b3e54284b0715d6fb23dd834)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:45.596360 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a23ae4-b808-489e-aa79-b7b241d76f6b admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:45.641870 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-d0a23ae4-b808-489e-aa79-b7b241d76f6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fef58a48b3e54284b0715d6fb23dd834'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:45.649049 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-d0a23ae4-b808-489e-aa79-b7b241d76f6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fef58a48b3e54284b0715d6fb23dd834'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:45.649667 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-d0a23ae4-b808-489e-aa79-b7b241d76f6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fef58a48b3e54284b0715d6fb23dd834'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:45.657517 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-d0a23ae4-b808-489e-aa79-b7b241d76f6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fef58a48b3e54284b0715d6fb23dd834'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:45.658173 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-d0a23ae4-b808-489e-aa79-b7b241d76f6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fef58a48b3e54284b0715d6fb23dd834'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:45.675670 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1342/2669] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:21:45 2022] DELETE /identity/v3/users/fef58a48b3e54284b0715d6fb23dd834 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:21:45.680175 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce797292-1ecc-46ea-b0cc-ac5dba8dcc4d None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:45.680651 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce797292-1ecc-46ea-b0cc-ac5dba8dcc4d None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:45.680759 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce797292-1ecc-46ea-b0cc-ac5dba8dcc4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:45.699002 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-ce797292-1ecc-46ea-b0cc-ac5dba8dcc4d None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:21:45.700225 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-ce797292-1ecc-46ea-b0cc-ac5dba8dcc4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:45.759031 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1328/2670] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:21:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:21:45.771953 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a23ae4-b808-489e-aa79-b7b241d76f6b admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:45.773369 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-d0a23ae4-b808-489e-aa79-b7b241d76f6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:45.830664 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a23ae4-b808-489e-aa79-b7b241d76f6b admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:45.834196 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed798e9f-c14e-4b91-84c9-97e82be4bfa7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:45.834816 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed798e9f-c14e-4b91-84c9-97e82be4bfa7 service neutron] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:45.835091 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed798e9f-c14e-4b91-84c9-97e82be4bfa7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:45.835400 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed798e9f-c14e-4b91-84c9-97e82be4bfa7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:45.845454 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed798e9f-c14e-4b91-84c9-97e82be4bfa7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:45.846484 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed798e9f-c14e-4b91-84c9-97e82be4bfa7 service neutron] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:45.856436 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1343/2671] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:21:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 12:21:46.206401 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce797292-1ecc-46ea-b0cc-ac5dba8dcc4d None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:46.208072 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-ce797292-1ecc-46ea-b0cc-ac5dba8dcc4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:46.221111 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce797292-1ecc-46ea-b0cc-ac5dba8dcc4d None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:46.225707 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87feff52-683f-4a4e-9102-8225a0e83f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:46.226528 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87feff52-683f-4a4e-9102-8225a0e83f3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:46.226826 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87feff52-683f-4a4e-9102-8225a0e83f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:46.227099 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87feff52-683f-4a4e-9102-8225a0e83f3b admin admin] PATH_INFO: `/v3/projects/30bf9bfe9e194ecbbfe5f16bfdf247d2` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:46.228256 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87feff52-683f-4a4e-9102-8225a0e83f3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=30bf9bfe9e194ecbbfe5f16bfdf247d2)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:46.230154 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87feff52-683f-4a4e-9102-8225a0e83f3b admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:46.281326 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-87feff52-683f-4a4e-9102-8225a0e83f3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30bf9bfe9e194ecbbfe5f16bfdf247d2', 'request_id': 'req-87feff52-683f-4a4e-9102-8225a0e83f3b'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:46.287596 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-87feff52-683f-4a4e-9102-8225a0e83f3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30bf9bfe9e194ecbbfe5f16bfdf247d2', 'request_id': 'req-87feff52-683f-4a4e-9102-8225a0e83f3b'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:46.294850 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-87feff52-683f-4a4e-9102-8225a0e83f3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30bf9bfe9e194ecbbfe5f16bfdf247d2', 'request_id': 'req-87feff52-683f-4a4e-9102-8225a0e83f3b'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:46.305191 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-87feff52-683f-4a4e-9102-8225a0e83f3b admin admin] The token cache is being invalidate because project 30bf9bfe9e194ecbbfe5f16bfdf247d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:21:46.305474 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-87feff52-683f-4a4e-9102-8225a0e83f3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:46.306087 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-87feff52-683f-4a4e-9102-8225a0e83f3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:46.307584 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1329/2672] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:21:46 2022] DELETE /identity/v3/projects/30bf9bfe9e194ecbbfe5f16bfdf247d2 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:21:46.313088 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a2fbe-9e3a-4ec0-bdd3-7edd87c382b6 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:46.313379 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a2fbe-9e3a-4ec0-bdd3-7edd87c382b6 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:46.313652 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a2fbe-9e3a-4ec0-bdd3-7edd87c382b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:46.334392 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-7f3a2fbe-9e3a-4ec0-bdd3-7edd87c382b6 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:21:46.335612 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-7f3a2fbe-9e3a-4ec0-bdd3-7edd87c382b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:46.376037 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:21:46.396162 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1344/2673] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:21:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:21:46.403834 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba271176-278a-4181-a04e-2ccb72e6e516 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:46.404119 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba271176-278a-4181-a04e-2ccb72e6e516 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:46.404393 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba271176-278a-4181-a04e-2ccb72e6e516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:46.413271 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:21:46.428057 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-ba271176-278a-4181-a04e-2ccb72e6e516 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:21:46.429250 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-ba271176-278a-4181-a04e-2ccb72e6e516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:46.437140 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1330/2674] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:21:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:21:46.441247 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3a2fbe-9e3a-4ec0-bdd3-7edd87c382b6 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:46.450692 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3a2fbe-9e3a-4ec0-bdd3-7edd87c382b6 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:46.455138 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ea6572-e48c-471e-b921-2b13b6e14434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:46.455751 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea6572-e48c-471e-b921-2b13b6e14434 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:46.456030 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea6572-e48c-471e-b921-2b13b6e14434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:46.456300 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea6572-e48c-471e-b921-2b13b6e14434 admin admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:46.456936 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ea6572-e48c-471e-b921-2b13b6e14434 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:21:46.457235 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ea6572-e48c-471e-b921-2b13b6e14434 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:46.458803 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ea6572-e48c-471e-b921-2b13b6e14434 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:46.471045 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1345/2675] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:21:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:21:46.475041 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba271176-278a-4181-a04e-2ccb72e6e516 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:46.484934 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba271176-278a-4181-a04e-2ccb72e6e516 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:46.489814 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2179d5dc-5fe2-48a8-bebf-11ba1ad3a7f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:46.490400 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2179d5dc-5fe2-48a8-bebf-11ba1ad3a7f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:46.490704 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2179d5dc-5fe2-48a8-bebf-11ba1ad3a7f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:46.490973 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2179d5dc-5fe2-48a8-bebf-11ba1ad3a7f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:46.491620 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2179d5dc-5fe2-48a8-bebf-11ba1ad3a7f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:46.493483 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2179d5dc-5fe2-48a8-bebf-11ba1ad3a7f5 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:46.519060 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1331/2676] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:21:46 2022] POST /identity/v3/projects => generated 385 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:21:46.523368 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ea6572-e48c-471e-b921-2b13b6e14434 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:46.535366 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ea6572-e48c-471e-b921-2b13b6e14434 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:46.590312 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e19e285-3bf5-42b8-a516-d85a51886070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:46.590902 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e19e285-3bf5-42b8-a516-d85a51886070 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:46.591174 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e19e285-3bf5-42b8-a516-d85a51886070 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:46.591470 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e19e285-3bf5-42b8-a516-d85a51886070 admin admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:46.592608 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e19e285-3bf5-42b8-a516-d85a51886070 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:46.594665 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e19e285-3bf5-42b8-a516-d85a51886070 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:46.615268 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1346/2677] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:21:46 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 12:21:46.619478 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2179d5dc-5fe2-48a8-bebf-11ba1ad3a7f5 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:46.629770 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2179d5dc-5fe2-48a8-bebf-11ba1ad3a7f5 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:46.634305 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09bc721-a0a1-497f-9a06-88cf8a408c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:46.634889 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09bc721-a0a1-497f-9a06-88cf8a408c08 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:46.635168 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09bc721-a0a1-497f-9a06-88cf8a408c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:46.635467 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09bc721-a0a1-497f-9a06-88cf8a408c08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:46.636093 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09bc721-a0a1-497f-9a06-88cf8a408c08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:46.637664 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09bc721-a0a1-497f-9a06-88cf8a408c08 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:46.648514 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1332/2678] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:21:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:21:46.652460 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e19e285-3bf5-42b8-a516-d85a51886070 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:46.662940 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e19e285-3bf5-42b8-a516-d85a51886070 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:46.667529 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d11bb5d-c47c-4ba1-b592-e0c263609213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:46.668111 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11bb5d-c47c-4ba1-b592-e0c263609213 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:46.668385 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11bb5d-c47c-4ba1-b592-e0c263609213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:46.668653 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11bb5d-c47c-4ba1-b592-e0c263609213 admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:46.669288 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d11bb5d-c47c-4ba1-b592-e0c263609213 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:46.670914 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d11bb5d-c47c-4ba1-b592-e0c263609213 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:46.681527 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1347/2679] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:21:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 12:21:46.685554 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09bc721-a0a1-497f-9a06-88cf8a408c08 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:46.695437 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09bc721-a0a1-497f-9a06-88cf8a408c08 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:46.699969 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b131a49e-0e26-43de-af68-756aa9d2d815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:46.700602 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b131a49e-0e26-43de-af68-756aa9d2d815 admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:46.700877 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b131a49e-0e26-43de-af68-756aa9d2d815 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:46.701146 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b131a49e-0e26-43de-af68-756aa9d2d815 admin admin] PATH_INFO: `/v3/projects/25c0ece0653d4f778dcbd402fb7def5f/users/a6edb68518fc406dbec800dcdc353b74/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:46.715707 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b131a49e-0e26-43de-af68-756aa9d2d815 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25c0ece0653d4f778dcbd402fb7def5f, user_id=a6edb68518fc406dbec800dcdc353b74, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:46.719029 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b131a49e-0e26-43de-af68-756aa9d2d815 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:46.732589 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1333/2680] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:21:46 2022] PUT /identity/v3/projects/25c0ece0653d4f778dcbd402fb7def5f/users/a6edb68518fc406dbec800dcdc353b74/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:21:46.800855 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f849773c-601d-415b-a77c-6bd2f7accd9f None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:46.801138 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f849773c-601d-415b-a77c-6bd2f7accd9f None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:46.801409 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f849773c-601d-415b-a77c-6bd2f7accd9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:46.828315 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-f849773c-601d-415b-a77c-6bd2f7accd9f None None] MFA Rules not processed for user `a6edb68518fc406dbec800dcdc353b74`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:21:46.829490 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-f849773c-601d-415b-a77c-6bd2f7accd9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:46.891248 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1348/2681] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:21:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:21:46.902460 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b131a49e-0e26-43de-af68-756aa9d2d815 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:46.904137 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-b131a49e-0e26-43de-af68-756aa9d2d815 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:46.960334 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b131a49e-0e26-43de-af68-756aa9d2d815 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:46.963995 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2064766-c5b8-4f40-b44b-7d169bcf8294 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:46.964593 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2064766-c5b8-4f40-b44b-7d169bcf8294 service cinder] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:46.964872 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2064766-c5b8-4f40-b44b-7d169bcf8294 service cinder] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:46.965143 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2064766-c5b8-4f40-b44b-7d169bcf8294 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:46.973961 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2064766-c5b8-4f40-b44b-7d169bcf8294 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:46.975412 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2064766-c5b8-4f40-b44b-7d169bcf8294 service cinder] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:46.983986 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1334/2682] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:21:46 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:21:49.549426 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f849773c-601d-415b-a77c-6bd2f7accd9f None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:49.591976 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:21:49.613608 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f849773c-601d-415b-a77c-6bd2f7accd9f None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:49.618305 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d358a0a-bf3a-47db-b60b-1dd48a4eddc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:49.618921 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d358a0a-bf3a-47db-b60b-1dd48a4eddc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:49.619197 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d358a0a-bf3a-47db-b60b-1dd48a4eddc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:49.619494 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d358a0a-bf3a-47db-b60b-1dd48a4eddc5 admin admin] PATH_INFO: `/v3/users/a6edb68518fc406dbec800dcdc353b74` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:49.620593 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d358a0a-bf3a-47db-b60b-1dd48a4eddc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6edb68518fc406dbec800dcdc353b74)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:49.622615 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d358a0a-bf3a-47db-b60b-1dd48a4eddc5 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:49.668296 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-7d358a0a-bf3a-47db-b60b-1dd48a4eddc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6edb68518fc406dbec800dcdc353b74'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:49.676366 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-7d358a0a-bf3a-47db-b60b-1dd48a4eddc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6edb68518fc406dbec800dcdc353b74'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:49.677084 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-7d358a0a-bf3a-47db-b60b-1dd48a4eddc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6edb68518fc406dbec800dcdc353b74'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:49.683965 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-7d358a0a-bf3a-47db-b60b-1dd48a4eddc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6edb68518fc406dbec800dcdc353b74'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:49.684593 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-7d358a0a-bf3a-47db-b60b-1dd48a4eddc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6edb68518fc406dbec800dcdc353b74'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:49.702234 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1349/2683] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:21:49 2022] DELETE /identity/v3/users/a6edb68518fc406dbec800dcdc353b74 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:21:49.706607 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a784a478-9f19-4cbd-a4be-06791771bd2a None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:49.707119 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a784a478-9f19-4cbd-a4be-06791771bd2a None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:49.707252 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a784a478-9f19-4cbd-a4be-06791771bd2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:49.725267 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-a784a478-9f19-4cbd-a4be-06791771bd2a None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:21:49.726439 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-a784a478-9f19-4cbd-a4be-06791771bd2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:49.784601 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1335/2684] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:21:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:21:49.794443 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d358a0a-bf3a-47db-b60b-1dd48a4eddc5 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:49.796408 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-7d358a0a-bf3a-47db-b60b-1dd48a4eddc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:49.852129 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d358a0a-bf3a-47db-b60b-1dd48a4eddc5 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:49.855729 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c83c65-4a64-4170-83f0-25fc89e86401 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:49.856357 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c83c65-4a64-4170-83f0-25fc89e86401 service neutron] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:49.856633 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c83c65-4a64-4170-83f0-25fc89e86401 service neutron] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:49.856903 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c83c65-4a64-4170-83f0-25fc89e86401 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:49.866488 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c83c65-4a64-4170-83f0-25fc89e86401 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:49.867927 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c83c65-4a64-4170-83f0-25fc89e86401 service neutron] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:49.877706 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1350/2685] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:21:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:21:50.147843 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a784a478-9f19-4cbd-a4be-06791771bd2a None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:50.149317 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-a784a478-9f19-4cbd-a4be-06791771bd2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:50.162402 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a784a478-9f19-4cbd-a4be-06791771bd2a None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:50.167705 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9950279-15ca-4e6d-9973-936f1c2b34a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:50.168341 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9950279-15ca-4e6d-9973-936f1c2b34a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:50.168617 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9950279-15ca-4e6d-9973-936f1c2b34a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:50.168885 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9950279-15ca-4e6d-9973-936f1c2b34a3 admin admin] PATH_INFO: `/v3/projects/25c0ece0653d4f778dcbd402fb7def5f` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:50.170002 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9950279-15ca-4e6d-9973-936f1c2b34a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25c0ece0653d4f778dcbd402fb7def5f)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:50.172711 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9950279-15ca-4e6d-9973-936f1c2b34a3 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:50.211553 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:21:50.219872 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-a9950279-15ca-4e6d-9973-936f1c2b34a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25c0ece0653d4f778dcbd402fb7def5f', 'request_id': 'req-a9950279-15ca-4e6d-9973-936f1c2b34a3'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:50.226311 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-a9950279-15ca-4e6d-9973-936f1c2b34a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25c0ece0653d4f778dcbd402fb7def5f', 'request_id': 'req-a9950279-15ca-4e6d-9973-936f1c2b34a3'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:50.234050 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-a9950279-15ca-4e6d-9973-936f1c2b34a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25c0ece0653d4f778dcbd402fb7def5f', 'request_id': 'req-a9950279-15ca-4e6d-9973-936f1c2b34a3'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:50.244329 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-a9950279-15ca-4e6d-9973-936f1c2b34a3 admin admin] The token cache is being invalidate because project 25c0ece0653d4f778dcbd402fb7def5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:21:50.244616 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-a9950279-15ca-4e6d-9973-936f1c2b34a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:50.245233 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-a9950279-15ca-4e6d-9973-936f1c2b34a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:50.246686 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1336/2686] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:21:50 2022] DELETE /identity/v3/projects/25c0ece0653d4f778dcbd402fb7def5f => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:21:50.251999 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c854a5-1b30-4f18-a6fa-28b169309d88 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:50.252308 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c854a5-1b30-4f18-a6fa-28b169309d88 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:50.252575 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c854a5-1b30-4f18-a6fa-28b169309d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:50.272785 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-58c854a5-1b30-4f18-a6fa-28b169309d88 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:21:50.273959 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-58c854a5-1b30-4f18-a6fa-28b169309d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:50.332249 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1351/2687] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:21:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:21:50.345569 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab2f85-e2a1-4176-b699-c550618924a0 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:50.346077 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab2f85-e2a1-4176-b699-c550618924a0 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:50.346498 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab2f85-e2a1-4176-b699-c550618924a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:50.367276 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-beab2f85-e2a1-4176-b699-c550618924a0 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:21:50.368480 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-beab2f85-e2a1-4176-b699-c550618924a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:50.376447 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1337/2688] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:21:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:21:50.380647 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c854a5-1b30-4f18-a6fa-28b169309d88 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:50.385419 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:21:50.392871 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c854a5-1b30-4f18-a6fa-28b169309d88 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:50.398267 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478a5040-a378-4e01-895f-2928fe87bfcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:50.398886 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a5040-a378-4e01-895f-2928fe87bfcb admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:50.399163 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a5040-a378-4e01-895f-2928fe87bfcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:50.399466 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a5040-a378-4e01-895f-2928fe87bfcb admin admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:50.400104 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478a5040-a378-4e01-895f-2928fe87bfcb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:21:50.400404 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478a5040-a378-4e01-895f-2928fe87bfcb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:50.401981 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478a5040-a378-4e01-895f-2928fe87bfcb admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:50.413970 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1352/2689] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:21:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:21:50.418163 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beab2f85-e2a1-4176-b699-c550618924a0 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:50.427943 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beab2f85-e2a1-4176-b699-c550618924a0 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:50.432597 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a84aeb-622c-4c0b-b4a0-faf0e58dc59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:50.433190 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a84aeb-622c-4c0b-b4a0-faf0e58dc59b admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:50.433468 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a84aeb-622c-4c0b-b4a0-faf0e58dc59b admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:50.433739 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a84aeb-622c-4c0b-b4a0-faf0e58dc59b admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:50.434354 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a84aeb-622c-4c0b-b4a0-faf0e58dc59b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:50.436334 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a84aeb-622c-4c0b-b4a0-faf0e58dc59b admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:50.463150 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1338/2690] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:21:50 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:21:50.467877 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478a5040-a378-4e01-895f-2928fe87bfcb admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:50.477729 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478a5040-a378-4e01-895f-2928fe87bfcb admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:50.533281 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273e961f-94e4-49bf-bc6b-73e5ef683357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:50.533870 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e961f-94e4-49bf-bc6b-73e5ef683357 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:50.534170 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e961f-94e4-49bf-bc6b-73e5ef683357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:50.534447 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e961f-94e4-49bf-bc6b-73e5ef683357 admin admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:50.535066 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273e961f-94e4-49bf-bc6b-73e5ef683357 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:50.536995 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273e961f-94e4-49bf-bc6b-73e5ef683357 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:50.557167 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1353/2691] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:21:50 2022] POST /identity/v3/users => generated 287 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 12:21:50.562510 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a84aeb-622c-4c0b-b4a0-faf0e58dc59b admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:50.572984 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a84aeb-622c-4c0b-b4a0-faf0e58dc59b admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:50.578389 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc8cf53-d9a7-44ea-9046-530c3007fcda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:50.578981 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc8cf53-d9a7-44ea-9046-530c3007fcda admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:50.579293 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc8cf53-d9a7-44ea-9046-530c3007fcda admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:50.579571 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc8cf53-d9a7-44ea-9046-530c3007fcda admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:50.580235 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc8cf53-d9a7-44ea-9046-530c3007fcda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:50.581820 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc8cf53-d9a7-44ea-9046-530c3007fcda admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:50.591641 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1339/2692] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:21:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 12:21:50.596337 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273e961f-94e4-49bf-bc6b-73e5ef683357 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:50.606160 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273e961f-94e4-49bf-bc6b-73e5ef683357 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:50.610832 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bebde9-a4af-4e01-8bcf-5a8da154ab20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:50.611450 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bebde9-a4af-4e01-8bcf-5a8da154ab20 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:50.611726 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bebde9-a4af-4e01-8bcf-5a8da154ab20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:50.612044 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bebde9-a4af-4e01-8bcf-5a8da154ab20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:50.612677 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bebde9-a4af-4e01-8bcf-5a8da154ab20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:50.614498 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bebde9-a4af-4e01-8bcf-5a8da154ab20 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:50.624970 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1354/2693] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:21:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 12:21:50.628900 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc8cf53-d9a7-44ea-9046-530c3007fcda admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:50.639516 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc8cf53-d9a7-44ea-9046-530c3007fcda admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:50.644562 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e23e8b1-22a1-43c4-bbee-3057026352a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:50.645302 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e23e8b1-22a1-43c4-bbee-3057026352a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:50.645650 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e23e8b1-22a1-43c4-bbee-3057026352a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:50.645990 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e23e8b1-22a1-43c4-bbee-3057026352a2 admin admin] PATH_INFO: `/v3/projects/f885c9fd8f394c999eee70e8d27180e0/users/4e949eeed6cd451d8dc5d0877963b260/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:50.661545 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e23e8b1-22a1-43c4-bbee-3057026352a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f885c9fd8f394c999eee70e8d27180e0, user_id=4e949eeed6cd451d8dc5d0877963b260, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:50.664500 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e23e8b1-22a1-43c4-bbee-3057026352a2 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:50.677638 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1340/2694] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:21:50 2022] PUT /identity/v3/projects/f885c9fd8f394c999eee70e8d27180e0/users/4e949eeed6cd451d8dc5d0877963b260/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:21:50.682916 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffa9621-d3f5-48ab-b9a4-1340b62718f4 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:50.683349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffa9621-d3f5-48ab-b9a4-1340b62718f4 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:50.683628 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffa9621-d3f5-48ab-b9a4-1340b62718f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:50.704145 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-bffa9621-d3f5-48ab-b9a4-1340b62718f4 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:21:50.705371 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-bffa9621-d3f5-48ab-b9a4-1340b62718f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:50.773426 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1355/2695] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:21:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:21:50.783029 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e23e8b1-22a1-43c4-bbee-3057026352a2 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:50.784761 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-8e23e8b1-22a1-43c4-bbee-3057026352a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:50.841294 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e23e8b1-22a1-43c4-bbee-3057026352a2 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:50.845121 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d913f68-8ddb-47df-924b-67fa210dfae8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:50.845721 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d913f68-8ddb-47df-924b-67fa210dfae8 service neutron] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:50.846000 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d913f68-8ddb-47df-924b-67fa210dfae8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:50.846272 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d913f68-8ddb-47df-924b-67fa210dfae8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:50.855146 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d913f68-8ddb-47df-924b-67fa210dfae8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:50.856601 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d913f68-8ddb-47df-924b-67fa210dfae8 service neutron] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:50.866138 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1341/2696] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:21:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:21:52.325812 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f10029-6bdb-4049-a19a-46fe802ab726 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:52.326339 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f10029-6bdb-4049-a19a-46fe802ab726 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:52.326573 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f10029-6bdb-4049-a19a-46fe802ab726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:52.353746 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-85f10029-6bdb-4049-a19a-46fe802ab726 None None] MFA Rules not processed for user `755b04cae45746569bfbf92f2f213a58`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:21:52.355122 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-85f10029-6bdb-4049-a19a-46fe802ab726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:52.437886 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1356/2697] 10.222.0.61 () {58 vars in 967 bytes} [Fri Aug 26 12:21:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:21:52.450670 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d913f68-8ddb-47df-924b-67fa210dfae8 service neutron] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:52.452362 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-0d913f68-8ddb-47df-924b-67fa210dfae8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:52.524520 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d913f68-8ddb-47df-924b-67fa210dfae8 service neutron] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:52.530102 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cda02d-3b05-4346-90ea-787dce44fb03 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead95a244d11432d9fe9f718e1a75f38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:52.530951 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cda02d-3b05-4346-90ea-787dce44fb03 service swift] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:52.531260 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cda02d-3b05-4346-90ea-787dce44fb03 service swift] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:52.531546 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cda02d-3b05-4346-90ea-787dce44fb03 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:52.538260 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:21:52.540729 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cda02d-3b05-4346-90ea-787dce44fb03 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:52.542716 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cda02d-3b05-4346-90ea-787dce44fb03 service swift] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:52.554619 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1342/2698] 10.222.0.61 () {60 vars in 1384 bytes} [Fri Aug 26 12:21:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 12:21:55.011110 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e482b89b-6ac7-48c4-9fef-4eac69ba67d9 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:55.011878 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e482b89b-6ac7-48c4-9fef-4eac69ba67d9 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:55.011878 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e482b89b-6ac7-48c4-9fef-4eac69ba67d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:55.042205 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:21:55.054523 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-e482b89b-6ac7-48c4-9fef-4eac69ba67d9 None None] MFA Rules not processed for user `4e949eeed6cd451d8dc5d0877963b260`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:21:55.055791 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-e482b89b-6ac7-48c4-9fef-4eac69ba67d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:55.134997 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1357/2699] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:21:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:21:55.139693 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cda02d-3b05-4346-90ea-787dce44fb03 service swift] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:55.152393 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cda02d-3b05-4346-90ea-787dce44fb03 service swift] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:55.157403 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a81eec0-e441-4d0c-84b9-9f02246024a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:55.158160 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a81eec0-e441-4d0c-84b9-9f02246024a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:55.158495 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a81eec0-e441-4d0c-84b9-9f02246024a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:55.158776 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a81eec0-e441-4d0c-84b9-9f02246024a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:55.159438 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a81eec0-e441-4d0c-84b9-9f02246024a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:55.161430 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a81eec0-e441-4d0c-84b9-9f02246024a9 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:55.196444 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1343/2700] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:21:55 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:21:55.200931 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e482b89b-6ac7-48c4-9fef-4eac69ba67d9 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:55.213204 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e482b89b-6ac7-48c4-9fef-4eac69ba67d9 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:55.292256 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f826574-7999-4fe0-a6a9-89624de42c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:55.292982 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f826574-7999-4fe0-a6a9-89624de42c76 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:55.293238 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f826574-7999-4fe0-a6a9-89624de42c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:55.293604 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f826574-7999-4fe0-a6a9-89624de42c76 admin admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:55.294283 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f826574-7999-4fe0-a6a9-89624de42c76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:55.297459 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f826574-7999-4fe0-a6a9-89624de42c76 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:55.319726 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1358/2701] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:21:55 2022] POST /identity/v3/users => generated 293 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:21:55.324052 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a81eec0-e441-4d0c-84b9-9f02246024a9 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:55.336029 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a81eec0-e441-4d0c-84b9-9f02246024a9 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:55.341708 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac948ec-c0c7-458c-9391-523be14fb99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:55.343087 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac948ec-c0c7-458c-9391-523be14fb99c admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:55.343331 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac948ec-c0c7-458c-9391-523be14fb99c admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:55.343553 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac948ec-c0c7-458c-9391-523be14fb99c admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:55.344322 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac948ec-c0c7-458c-9391-523be14fb99c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:55.346252 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac948ec-c0c7-458c-9391-523be14fb99c admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:55.360578 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1344/2702] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:21:55 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 12:21:55.365010 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f826574-7999-4fe0-a6a9-89624de42c76 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:55.376066 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f826574-7999-4fe0-a6a9-89624de42c76 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:55.381049 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b485b749-ef5b-47a1-a6e8-285dd6962707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:55.381743 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485b749-ef5b-47a1-a6e8-285dd6962707 admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:55.382021 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485b749-ef5b-47a1-a6e8-285dd6962707 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:55.382298 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485b749-ef5b-47a1-a6e8-285dd6962707 admin admin] PATH_INFO: `/v3/projects/c7fa9fc32592449ebfda1144f8887475/users/f9bc3cca053e402e819e386d57b66a7c/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:55.397995 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b485b749-ef5b-47a1-a6e8-285dd6962707 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7fa9fc32592449ebfda1144f8887475, user_id=f9bc3cca053e402e819e386d57b66a7c, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:55.400684 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b485b749-ef5b-47a1-a6e8-285dd6962707 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:55.414814 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1359/2703] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:21:55 2022] PUT /identity/v3/projects/c7fa9fc32592449ebfda1144f8887475/users/f9bc3cca053e402e819e386d57b66a7c/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:21:55.421170 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac948ec-c0c7-458c-9391-523be14fb99c admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:55.432041 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac948ec-c0c7-458c-9391-523be14fb99c admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:55.492362 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8730b0a8-e22f-44fa-b24d-890ef529d716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:55.492976 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8730b0a8-e22f-44fa-b24d-890ef529d716 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:55.493261 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8730b0a8-e22f-44fa-b24d-890ef529d716 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:55.493535 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8730b0a8-e22f-44fa-b24d-890ef529d716 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:55.494180 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8730b0a8-e22f-44fa-b24d-890ef529d716 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:55.495951 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8730b0a8-e22f-44fa-b24d-890ef529d716 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:55.506814 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1345/2704] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:21:55 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:21:55.510795 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b485b749-ef5b-47a1-a6e8-285dd6962707 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:55.522026 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b485b749-ef5b-47a1-a6e8-285dd6962707 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:55.526740 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6f4da3-164f-4690-bd18-05324accb8a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:55.527434 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f4da3-164f-4690-bd18-05324accb8a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:55.527717 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f4da3-164f-4690-bd18-05324accb8a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:55.527993 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f4da3-164f-4690-bd18-05324accb8a9 admin admin] PATH_INFO: `/v3/projects/c7fa9fc32592449ebfda1144f8887475/users/f9bc3cca053e402e819e386d57b66a7c/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:55.530874 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6f4da3-164f-4690-bd18-05324accb8a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7fa9fc32592449ebfda1144f8887475, user_id=f9bc3cca053e402e819e386d57b66a7c, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:55.533580 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6f4da3-164f-4690-bd18-05324accb8a9 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:55.544619 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1360/2705] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:21:55 2022] PUT /identity/v3/projects/c7fa9fc32592449ebfda1144f8887475/users/f9bc3cca053e402e819e386d57b66a7c/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:21:58.539314 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8730b0a8-e22f-44fa-b24d-890ef529d716 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:58.543279 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-8730b0a8-e22f-44fa-b24d-890ef529d716 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:58.570516 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8730b0a8-e22f-44fa-b24d-890ef529d716 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:58.661063 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0da926-f05a-4d77-ba96-db4ee7dd4eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:58.662130 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0da926-f05a-4d77-ba96-db4ee7dd4eb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:58.662646 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0da926-f05a-4d77-ba96-db4ee7dd4eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:58.663093 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0da926-f05a-4d77-ba96-db4ee7dd4eb3 admin admin] PATH_INFO: `/v3/users/e0ac29547caa4204b7f30ebb143e0032` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:58.665179 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0da926-f05a-4d77-ba96-db4ee7dd4eb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0ac29547caa4204b7f30ebb143e0032)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:58.667958 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0da926-f05a-4d77-ba96-db4ee7dd4eb3 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:58.772345 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-8b0da926-f05a-4d77-ba96-db4ee7dd4eb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0ac29547caa4204b7f30ebb143e0032'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:58.793121 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-8b0da926-f05a-4d77-ba96-db4ee7dd4eb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0ac29547caa4204b7f30ebb143e0032'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:58.801486 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:21:58.803897 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-8b0da926-f05a-4d77-ba96-db4ee7dd4eb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0ac29547caa4204b7f30ebb143e0032'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:58.804765 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-8b0da926-f05a-4d77-ba96-db4ee7dd4eb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0ac29547caa4204b7f30ebb143e0032'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:58.823289 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-8b0da926-f05a-4d77-ba96-db4ee7dd4eb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0ac29547caa4204b7f30ebb143e0032'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:58.845383 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1346/2706] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:21:58 2022] DELETE /identity/v3/users/e0ac29547caa4204b7f30ebb143e0032 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:21:58.850272 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd24125e-a06c-437c-bea3-2e0a17085691 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:58.851191 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd24125e-a06c-437c-bea3-2e0a17085691 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:58.851682 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd24125e-a06c-437c-bea3-2e0a17085691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:58.880420 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-cd24125e-a06c-437c-bea3-2e0a17085691 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:21:58.881973 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-cd24125e-a06c-437c-bea3-2e0a17085691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:58.947665 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:21:58.973276 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1361/2707] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:21:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:21:58.987600 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0da926-f05a-4d77-ba96-db4ee7dd4eb3 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:58.991000 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-8b0da926-f05a-4d77-ba96-db4ee7dd4eb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:59.095470 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0da926-f05a-4d77-ba96-db4ee7dd4eb3 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:59.099286 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc36b17-75b8-482d-819f-956b2c9c2c4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:59.100060 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc36b17-75b8-482d-819f-956b2c9c2c4d service neutron] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:59.100342 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc36b17-75b8-482d-819f-956b2c9c2c4d service neutron] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:59.100612 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc36b17-75b8-482d-819f-956b2c9c2c4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:59.129631 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc36b17-75b8-482d-819f-956b2c9c2c4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:59.134229 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc36b17-75b8-482d-819f-956b2c9c2c4d service neutron] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:59.157167 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1347/2708] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:21:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 12:21:59.525865 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd24125e-a06c-437c-bea3-2e0a17085691 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:21:59.527542 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-cd24125e-a06c-437c-bea3-2e0a17085691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:21:59.554713 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd24125e-a06c-437c-bea3-2e0a17085691 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:21:59.568161 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20c6d99-aa9a-4e0f-a521-3d22f2c36580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:21:59.569033 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20c6d99-aa9a-4e0f-a521-3d22f2c36580 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:21:59.569462 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20c6d99-aa9a-4e0f-a521-3d22f2c36580 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:21:59.569913 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20c6d99-aa9a-4e0f-a521-3d22f2c36580 admin admin] PATH_INFO: `/v3/projects/50e8bcf90bd149589df963d40ad79f32` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:21:59.571365 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20c6d99-aa9a-4e0f-a521-3d22f2c36580 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50e8bcf90bd149589df963d40ad79f32)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:21:59.573596 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20c6d99-aa9a-4e0f-a521-3d22f2c36580 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:21:59.656048 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-e20c6d99-aa9a-4e0f-a521-3d22f2c36580 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50e8bcf90bd149589df963d40ad79f32', 'request_id': 'req-e20c6d99-aa9a-4e0f-a521-3d22f2c36580'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:59.664623 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-e20c6d99-aa9a-4e0f-a521-3d22f2c36580 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50e8bcf90bd149589df963d40ad79f32', 'request_id': 'req-e20c6d99-aa9a-4e0f-a521-3d22f2c36580'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:59.679232 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-e20c6d99-aa9a-4e0f-a521-3d22f2c36580 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50e8bcf90bd149589df963d40ad79f32', 'request_id': 'req-e20c6d99-aa9a-4e0f-a521-3d22f2c36580'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:59.703599 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-e20c6d99-aa9a-4e0f-a521-3d22f2c36580 admin admin] The token cache is being invalidate because project 50e8bcf90bd149589df963d40ad79f32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81712) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:21:59.703952 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-e20c6d99-aa9a-4e0f-a521-3d22f2c36580 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:59.704578 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-e20c6d99-aa9a-4e0f-a521-3d22f2c36580 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:21:59.706401 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1362/2709] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:21:59 2022] DELETE /identity/v3/projects/50e8bcf90bd149589df963d40ad79f32 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:22:00.054774 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ca274-077a-40fd-b0f6-d95f9e1536b0 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:22:00.055413 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ca274-077a-40fd-b0f6-d95f9e1536b0 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:22:00.055413 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ca274-077a-40fd-b0f6-d95f9e1536b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:22:00.085830 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-3b2ca274-077a-40fd-b0f6-d95f9e1536b0 None None] MFA Rules not processed for user `f9bc3cca053e402e819e386d57b66a7c`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:22:00.087080 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-3b2ca274-077a-40fd-b0f6-d95f9e1536b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:22:00.159908 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1348/2710] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:22:00 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:22:00.173784 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20c6d99-aa9a-4e0f-a521-3d22f2c36580 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:22:00.175185 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-e20c6d99-aa9a-4e0f-a521-3d22f2c36580 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:22:00.266230 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20c6d99-aa9a-4e0f-a521-3d22f2c36580 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:22:00.274215 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a0def1-b1cb-4f96-b3f4-948d2dc4517c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aacd7d6d17e4f10ba5c9107a12beafa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:22:00.275067 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a0def1-b1cb-4f96-b3f4-948d2dc4517c service nova] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:22:00.275393 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a0def1-b1cb-4f96-b3f4-948d2dc4517c service nova] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:22:00.275671 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a0def1-b1cb-4f96-b3f4-948d2dc4517c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:22:00.277401 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-b4a0def1-b1cb-4f96-b3f4-948d2dc4517c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:22:00.343532 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a0def1-b1cb-4f96-b3f4-948d2dc4517c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:22:00.345119 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a0def1-b1cb-4f96-b3f4-948d2dc4517c service nova] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:22:00.354473 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1363/2711] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:22:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:22:01.425372 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ca0b3-6130-4e3a-a2d7-bf9154b4c2af None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:22:01.425846 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ca0b3-6130-4e3a-a2d7-bf9154b4c2af None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:22:01.425940 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ca0b3-6130-4e3a-a2d7-bf9154b4c2af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:22:01.466959 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-6e3ca0b3-6130-4e3a-a2d7-bf9154b4c2af None None] MFA Rules not processed for user `4e949eeed6cd451d8dc5d0877963b260`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:22:01.466959 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-6e3ca0b3-6130-4e3a-a2d7-bf9154b4c2af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:22:01.466959 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1349/2712] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:22:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:22:10.350031 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=85,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:22:11.439841 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=71,keystone:DELETE=2 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:22:35.317416 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a0def1-b1cb-4f96-b3f4-948d2dc4517c service nova] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:22:35.318777 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-b4a0def1-b1cb-4f96-b3f4-948d2dc4517c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:22:35.423093 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a0def1-b1cb-4f96-b3f4-948d2dc4517c service nova] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:22:35.426700 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05709f91-1307-4b95-a17c-427f81daf977 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:22:35.427326 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05709f91-1307-4b95-a17c-427f81daf977 service neutron] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:22:35.427605 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05709f91-1307-4b95-a17c-427f81daf977 service neutron] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:22:35.427875 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05709f91-1307-4b95-a17c-427f81daf977 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:22:35.456477 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05709f91-1307-4b95-a17c-427f81daf977 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:22:35.457896 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05709f91-1307-4b95-a17c-427f81daf977 service neutron] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:22:35.467074 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1364/2713] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:22:35 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:22:37.981775 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3ca0b3-6130-4e3a-a2d7-bf9154b4c2af None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:22:37.983505 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-6e3ca0b3-6130-4e3a-a2d7-bf9154b4c2af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:22:38.135801 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3ca0b3-6130-4e3a-a2d7-bf9154b4c2af None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:22:38.142101 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b71e3ae-d5d7-4586-ad8b-343c04241e3c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7346e4a29dfb4a0a9768597704be286c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:22:38.142734 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71e3ae-d5d7-4586-ad8b-343c04241e3c service placement] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:22:38.143487 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71e3ae-d5d7-4586-ad8b-343c04241e3c service placement] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:22:38.143687 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71e3ae-d5d7-4586-ad8b-343c04241e3c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:22:38.145695 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-5b71e3ae-d5d7-4586-ad8b-343c04241e3c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:22:38.158392 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b71e3ae-d5d7-4586-ad8b-343c04241e3c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:22:38.160051 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b71e3ae-d5d7-4586-ad8b-343c04241e3c service placement] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:22:38.171366 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1350/2714] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:22:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:22:41.439690 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c1655-4ad8-46d7-ad08-4a5e9b8bc015 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:22:41.440211 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c1655-4ad8-46d7-ad08-4a5e9b8bc015 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:22:41.440328 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c1655-4ad8-46d7-ad08-4a5e9b8bc015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:22:41.459651 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-715c1655-4ad8-46d7-ad08-4a5e9b8bc015 None None] MFA Rules not processed for user `0e013faf32b84179989eb99be958555f`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:22:41.460834 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-715c1655-4ad8-46d7-ad08-4a5e9b8bc015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:22:41.513736 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1365/2715] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:22:41 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:22:48.166859 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=53 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:22:51.507610 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=82 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:23:45.625831 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331aa25e-dcf2-4d2f-b036-24abaf5e81ed None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:23:45.626324 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331aa25e-dcf2-4d2f-b036-24abaf5e81ed None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:23:45.626434 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331aa25e-dcf2-4d2f-b036-24abaf5e81ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:23:45.654615 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-331aa25e-dcf2-4d2f-b036-24abaf5e81ed None None] MFA Rules not processed for user `755b04cae45746569bfbf92f2f213a58`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:23:45.655902 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-331aa25e-dcf2-4d2f-b036-24abaf5e81ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:23:45.717077 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1351/2716] 10.222.0.61 () {58 vars in 1009 bytes} [Fri Aug 26 12:23:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:23:45.764584 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715c1655-4ad8-46d7-ad08-4a5e9b8bc015 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:23:45.765981 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-715c1655-4ad8-46d7-ad08-4a5e9b8bc015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:23:45.834190 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715c1655-4ad8-46d7-ad08-4a5e9b8bc015 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:23:45.839052 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a15449-144b-4d20-b062-1a385da2875c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead95a244d11432d9fe9f718e1a75f38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:23:45.839675 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a15449-144b-4d20-b062-1a385da2875c service swift] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:23:45.839950 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a15449-144b-4d20-b062-1a385da2875c service swift] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:23:45.840217 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a15449-144b-4d20-b062-1a385da2875c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:23:45.848275 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a15449-144b-4d20-b062-1a385da2875c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:23:45.849826 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a15449-144b-4d20-b062-1a385da2875c service swift] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:23:45.858087 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1366/2717] 10.222.0.61 () {60 vars in 1384 bytes} [Fri Aug 26 12:23:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 12:23:55.710728 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=32 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:23:55.854461 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=32 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:23:58.888019 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e11a649-b9d9-4411-9837-2cfcdea826da None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:23:58.888681 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e11a649-b9d9-4411-9837-2cfcdea826da None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:23:58.888681 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e11a649-b9d9-4411-9837-2cfcdea826da None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:23:58.890480 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1352/2718] 10.222.0.61 () {54 vars in 921 bytes} [Fri Aug 26 12:23:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:23:58.897981 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020697e0-5a8d-4ae9-94b7-d4938a415a16 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:23:58.898545 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020697e0-5a8d-4ae9-94b7-d4938a415a16 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:23:58.898664 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020697e0-5a8d-4ae9-94b7-d4938a415a16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:23:58.942380 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-020697e0-5a8d-4ae9-94b7-d4938a415a16 None None] MFA Rules not processed for user `7aacd7d6d17e4f10ba5c9107a12beafa`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:23:58.943636 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-020697e0-5a8d-4ae9-94b7-d4938a415a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:23:58.962143 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1367/2719] 10.222.0.61 () {58 vars in 1018 bytes} [Fri Aug 26 12:23:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:23:58.974008 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e11a649-b9d9-4411-9837-2cfcdea826da None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:23:58.975727 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-8e11a649-b9d9-4411-9837-2cfcdea826da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:23:59.040772 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e11a649-b9d9-4411-9837-2cfcdea826da None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:23:59.044361 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393f3656-bc58-475f-8ee3-75f6e7be440d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '152fd7c708114f249b9c960165f0fdfe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:23:59.044970 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f3656-bc58-475f-8ee3-75f6e7be440d service glance] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:23:59.045248 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f3656-bc58-475f-8ee3-75f6e7be440d service glance] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:23:59.045518 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393f3656-bc58-475f-8ee3-75f6e7be440d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:23:59.055518 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393f3656-bc58-475f-8ee3-75f6e7be440d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:23:59.056938 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393f3656-bc58-475f-8ee3-75f6e7be440d service glance] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:23:59.067760 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1353/2720] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:23:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:23:59.136162 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5609c3bf-09c3-4e8f-bb72-d167d517dfa0 None None] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:23:59.136544 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5609c3bf-09c3-4e8f-bb72-d167d517dfa0 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:23:59.136893 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5609c3bf-09c3-4e8f-bb72-d167d517dfa0 None None] PATH_INFO: `/` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:23:59.138306 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1368/2721] 10.222.0.61 () {54 vars in 921 bytes} [Fri Aug 26 12:23:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:23:59.143479 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e1cfc-2ed5-4b58-8031-4fdf169bc393 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:23:59.143768 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e1cfc-2ed5-4b58-8031-4fdf169bc393 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:23:59.144046 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e1cfc-2ed5-4b58-8031-4fdf169bc393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:23:59.163792 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-692e1cfc-2ed5-4b58-8031-4fdf169bc393 None None] MFA Rules not processed for user `b0972dbd5cff4610b239326425e6351c`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:23:59.164980 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-692e1cfc-2ed5-4b58-8031-4fdf169bc393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:23:59.171588 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1354/2722] 10.222.0.61 () {58 vars in 1018 bytes} [Fri Aug 26 12:23:59 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:23:59.183068 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5609c3bf-09c3-4e8f-bb72-d167d517dfa0 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:23:59.193900 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5609c3bf-09c3-4e8f-bb72-d167d517dfa0 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:23:59.197411 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c2deaa-6299-4c5e-8608-1a1c6cfbf402 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:23:59.197995 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c2deaa-6299-4c5e-8608-1a1c6cfbf402 service neutron] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:23:59.198272 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c2deaa-6299-4c5e-8608-1a1c6cfbf402 service neutron] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:23:59.198543 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c2deaa-6299-4c5e-8608-1a1c6cfbf402 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:23:59.206217 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c2deaa-6299-4c5e-8608-1a1c6cfbf402 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:23:59.207677 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c2deaa-6299-4c5e-8608-1a1c6cfbf402 service neutron] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:23:59.215300 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1369/2723] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:23:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:23:59.299023 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54284d4-e5a9-411d-b8cc-e8d47624976c None None] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:23:59.299278 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54284d4-e5a9-411d-b8cc-e8d47624976c None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:23:59.299556 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54284d4-e5a9-411d-b8cc-e8d47624976c None None] PATH_INFO: `/` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:23:59.300896 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1355/2724] 10.222.0.61 () {54 vars in 921 bytes} [Fri Aug 26 12:23:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 26 12:23:59.305558 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10053a6b-1d35-413b-a725-17b5886589a0 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:23:59.305841 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10053a6b-1d35-413b-a725-17b5886589a0 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:23:59.306115 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10053a6b-1d35-413b-a725-17b5886589a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:23:59.325462 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-10053a6b-1d35-413b-a725-17b5886589a0 None None] MFA Rules not processed for user `7aacd7d6d17e4f10ba5c9107a12beafa`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:23:59.327161 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-10053a6b-1d35-413b-a725-17b5886589a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:23:59.336300 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1370/2725] 10.222.0.61 () {58 vars in 1018 bytes} [Fri Aug 26 12:23:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:23:59.348064 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54284d4-e5a9-411d-b8cc-e8d47624976c None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:23:59.349749 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-b54284d4-e5a9-411d-b8cc-e8d47624976c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:23:59.363855 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54284d4-e5a9-411d-b8cc-e8d47624976c None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:23:59.367554 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7a7100-820d-47f9-a09c-ef945754267f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:23:59.368184 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7a7100-820d-47f9-a09c-ef945754267f service neutron] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:23:59.368464 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7a7100-820d-47f9-a09c-ef945754267f service neutron] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:23:59.368739 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7a7100-820d-47f9-a09c-ef945754267f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:23:59.379372 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7a7100-820d-47f9-a09c-ef945754267f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:23:59.380787 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7a7100-820d-47f9-a09c-ef945754267f service neutron] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:23:59.392534 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1356/2726] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:23:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:24:01.442341 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a795fab-33a0-4d4d-b828-b2f05ba658c8 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:24:01.442872 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a795fab-33a0-4d4d-b828-b2f05ba658c8 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:24:01.443664 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a795fab-33a0-4d4d-b828-b2f05ba658c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:24:01.461534 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-2a795fab-33a0-4d4d-b828-b2f05ba658c8 None None] MFA Rules not processed for user `755b04cae45746569bfbf92f2f213a58`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:24:01.462690 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-2a795fab-33a0-4d4d-b828-b2f05ba658c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:24:01.470119 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1371/2727] 10.222.0.61 () {58 vars in 1009 bytes} [Fri Aug 26 12:24:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:24:01.481756 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7a7100-820d-47f9-a09c-ef945754267f service neutron] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:24:01.493830 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7a7100-820d-47f9-a09c-ef945754267f service neutron] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:24:01.499642 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b53729-0da7-4e04-9b37-74fe57a1388e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead95a244d11432d9fe9f718e1a75f38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:24:01.500236 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b53729-0da7-4e04-9b37-74fe57a1388e service swift] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:24:01.500514 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b53729-0da7-4e04-9b37-74fe57a1388e service swift] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:24:01.500782 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b53729-0da7-4e04-9b37-74fe57a1388e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:24:01.508819 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b53729-0da7-4e04-9b37-74fe57a1388e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:24:01.510380 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b53729-0da7-4e04-9b37-74fe57a1388e service swift] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:24:01.518337 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1357/2728] 10.222.0.61 () {60 vars in 1384 bytes} [Fri Aug 26 12:24:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 12:24:08.416716 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a795fab-33a0-4d4d-b828-b2f05ba658c8 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:24:08.429842 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a795fab-33a0-4d4d-b828-b2f05ba658c8 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:24:08.433777 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f9e133-3c28-494c-9b95-5b27ce04ffb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:24:08.434413 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f9e133-3c28-494c-9b95-5b27ce04ffb4 service neutron] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:24:08.434686 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f9e133-3c28-494c-9b95-5b27ce04ffb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:24:08.434951 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f9e133-3c28-494c-9b95-5b27ce04ffb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:24:08.437227 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-e3f9e133-3c28-494c-9b95-5b27ce04ffb4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:24:08.451223 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f9e133-3c28-494c-9b95-5b27ce04ffb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:24:08.452641 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f9e133-3c28-494c-9b95-5b27ce04ffb4 service neutron] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:24:08.463307 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1372/2729] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:24:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:24:11.515243 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=41 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:24:12.022441 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b53729-0da7-4e04-9b37-74fe57a1388e service swift] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:24:12.032712 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b53729-0da7-4e04-9b37-74fe57a1388e service swift] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:24:12.036236 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29cf5e-8cb3-4985-8a8a-da7f89201b19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:24:12.036825 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29cf5e-8cb3-4985-8a8a-da7f89201b19 service neutron] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:24:12.037099 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29cf5e-8cb3-4985-8a8a-da7f89201b19 service neutron] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:24:12.037365 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb29cf5e-8cb3-4985-8a8a-da7f89201b19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:24:12.039112 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-fb29cf5e-8cb3-4985-8a8a-da7f89201b19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:24:12.053166 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb29cf5e-8cb3-4985-8a8a-da7f89201b19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:24:12.054555 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb29cf5e-8cb3-4985-8a8a-da7f89201b19 service neutron] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:24:12.065543 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1358/2730] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:24:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:24:16.013746 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602a2511-dd04-49d2-804f-d35798703624 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:24:16.014245 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602a2511-dd04-49d2-804f-d35798703624 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:24:16.014342 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602a2511-dd04-49d2-804f-d35798703624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:24:16.034021 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-602a2511-dd04-49d2-804f-d35798703624 None None] MFA Rules not processed for user `755b04cae45746569bfbf92f2f213a58`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:24:16.035248 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-602a2511-dd04-49d2-804f-d35798703624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:24:16.042209 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1373/2731] 10.222.0.61 () {58 vars in 967 bytes} [Fri Aug 26 12:24:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:24:16.054593 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29cf5e-8cb3-4985-8a8a-da7f89201b19 service neutron] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:24:16.066488 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb29cf5e-8cb3-4985-8a8a-da7f89201b19 service neutron] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:24:16.071547 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411a86a8-6f67-4cba-a57d-2b614f20aeeb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead95a244d11432d9fe9f718e1a75f38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:24:16.072135 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411a86a8-6f67-4cba-a57d-2b614f20aeeb service swift] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:24:16.072411 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411a86a8-6f67-4cba-a57d-2b614f20aeeb service swift] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:24:16.072677 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411a86a8-6f67-4cba-a57d-2b614f20aeeb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:24:16.080985 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411a86a8-6f67-4cba-a57d-2b614f20aeeb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:24:16.082542 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411a86a8-6f67-4cba-a57d-2b614f20aeeb service swift] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:24:16.090338 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1359/2732] 10.222.0.61 () {60 vars in 1384 bytes} [Fri Aug 26 12:24:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 12:24:26.029382 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=37 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:24:26.087207 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=12 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:24:43.958214 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602a2511-dd04-49d2-804f-d35798703624 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:24:43.968252 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602a2511-dd04-49d2-804f-d35798703624 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:24:43.971728 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2183205-b712-43b2-ac85-87e40200058b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:24:43.972340 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2183205-b712-43b2-ac85-87e40200058b service neutron] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:24:43.972613 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2183205-b712-43b2-ac85-87e40200058b service neutron] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:24:43.972879 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2183205-b712-43b2-ac85-87e40200058b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:24:43.974573 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-a2183205-b712-43b2-ac85-87e40200058b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:24:43.984513 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2183205-b712-43b2-ac85-87e40200058b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:24:43.986157 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2183205-b712-43b2-ac85-87e40200058b service neutron] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:24:43.994223 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1374/2733] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:24:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:24:46.882237 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411a86a8-6f67-4cba-a57d-2b614f20aeeb service swift] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:24:46.892262 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411a86a8-6f67-4cba-a57d-2b614f20aeeb service swift] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:24:46.895835 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6a6b67-7adc-4941-af23-e633e0b331bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:24:46.896417 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6a6b67-7adc-4941-af23-e633e0b331bd service neutron] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:24:46.896694 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6a6b67-7adc-4941-af23-e633e0b331bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:24:46.896963 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6a6b67-7adc-4941-af23-e633e0b331bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:24:46.898642 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-3c6a6b67-7adc-4941-af23-e633e0b331bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:24:46.909091 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6a6b67-7adc-4941-af23-e633e0b331bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:24:46.910573 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6a6b67-7adc-4941-af23-e633e0b331bd service neutron] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:24:46.918550 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1360/2734] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:24:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:24:47.213247 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2183205-b712-43b2-ac85-87e40200058b service neutron] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:24:47.225756 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2183205-b712-43b2-ac85-87e40200058b service neutron] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:24:47.230652 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d4ffa5-2fbe-4009-a724-d6ce970beb9e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7346e4a29dfb4a0a9768597704be286c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:24:47.231271 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d4ffa5-2fbe-4009-a724-d6ce970beb9e service placement] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:24:47.231547 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d4ffa5-2fbe-4009-a724-d6ce970beb9e service placement] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:24:47.231810 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d4ffa5-2fbe-4009-a724-d6ce970beb9e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:24:47.234345 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-36d4ffa5-2fbe-4009-a724-d6ce970beb9e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:24:47.250054 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d4ffa5-2fbe-4009-a724-d6ce970beb9e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:24:47.251662 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d4ffa5-2fbe-4009-a724-d6ce970beb9e service placement] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:24:47.261590 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1375/2735] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:24:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:24:50.451948 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6a6b67-7adc-4941-af23-e633e0b331bd service neutron] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:24:50.460516 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6a6b67-7adc-4941-af23-e633e0b331bd service neutron] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:24:50.464168 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdde3695-c2ec-4a1a-b766-c7205e400645 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:24:50.464737 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdde3695-c2ec-4a1a-b766-c7205e400645 service neutron] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:24:50.465017 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdde3695-c2ec-4a1a-b766-c7205e400645 service neutron] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:24:50.465286 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdde3695-c2ec-4a1a-b766-c7205e400645 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:24:50.467183 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-fdde3695-c2ec-4a1a-b766-c7205e400645 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:24:50.478105 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdde3695-c2ec-4a1a-b766-c7205e400645 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:24:50.479542 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdde3695-c2ec-4a1a-b766-c7205e400645 service neutron] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:24:50.489401 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1361/2736] 10.222.0.61 () {60 vars in 1396 bytes} [Fri Aug 26 12:24:50 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3266 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:24:53.086306 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d4ffa5-2fbe-4009-a724-d6ce970beb9e service placement] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:24:53.105600 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d4ffa5-2fbe-4009-a724-d6ce970beb9e service placement] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:24:53.113380 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c71bd7-480b-41f6-8af0-e80c3092dca0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aacd7d6d17e4f10ba5c9107a12beafa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:24:53.114571 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c71bd7-480b-41f6-8af0-e80c3092dca0 service nova] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:24:53.115083 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c71bd7-480b-41f6-8af0-e80c3092dca0 service nova] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:24:53.115745 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c71bd7-480b-41f6-8af0-e80c3092dca0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:24:53.118490 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-58c71bd7-480b-41f6-8af0-e80c3092dca0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:24:53.134108 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c71bd7-480b-41f6-8af0-e80c3092dca0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:24:53.136412 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c71bd7-480b-41f6-8af0-e80c3092dca0 service nova] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:24:53.160937 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1376/2737] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:24:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:24:57.125557 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdde3695-c2ec-4a1a-b766-c7205e400645 service neutron] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:24:57.134308 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdde3695-c2ec-4a1a-b766-c7205e400645 service neutron] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:24:57.138041 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8836d5-ffb7-4466-9be6-6bab16b5326d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:24:57.138712 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8836d5-ffb7-4466-9be6-6bab16b5326d service neutron] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:24:57.139733 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8836d5-ffb7-4466-9be6-6bab16b5326d service neutron] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:24:57.140084 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8836d5-ffb7-4466-9be6-6bab16b5326d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:24:57.141907 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-6c8836d5-ffb7-4466-9be6-6bab16b5326d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:24:57.230562 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8836d5-ffb7-4466-9be6-6bab16b5326d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:24:57.232030 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8836d5-ffb7-4466-9be6-6bab16b5326d service neutron] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:24:57.257831 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1362/2738] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:24:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:25:00.432942 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c71bd7-480b-41f6-8af0-e80c3092dca0 service nova] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:00.436601 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-58c71bd7-480b-41f6-8af0-e80c3092dca0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:00.465418 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c71bd7-480b-41f6-8af0-e80c3092dca0 service nova] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:00.470878 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012351e2-5bdb-4cd9-8beb-18741535d563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:00.471656 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012351e2-5bdb-4cd9-8beb-18741535d563 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:00.472101 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012351e2-5bdb-4cd9-8beb-18741535d563 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:00.472560 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012351e2-5bdb-4cd9-8beb-18741535d563 admin admin] PATH_INFO: `/v3/users/0e013faf32b84179989eb99be958555f` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:00.474393 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012351e2-5bdb-4cd9-8beb-18741535d563 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e013faf32b84179989eb99be958555f)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:00.476641 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012351e2-5bdb-4cd9-8beb-18741535d563 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:00.534039 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-012351e2-5bdb-4cd9-8beb-18741535d563 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e013faf32b84179989eb99be958555f'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:00.541758 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-012351e2-5bdb-4cd9-8beb-18741535d563 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e013faf32b84179989eb99be958555f'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:00.542831 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-012351e2-5bdb-4cd9-8beb-18741535d563 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e013faf32b84179989eb99be958555f'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:00.550753 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-012351e2-5bdb-4cd9-8beb-18741535d563 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e013faf32b84179989eb99be958555f'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:00.551461 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-012351e2-5bdb-4cd9-8beb-18741535d563 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e013faf32b84179989eb99be958555f'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:00.572946 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1377/2739] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:25:00 2022] DELETE /identity/v3/users/0e013faf32b84179989eb99be958555f => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:25:00.751961 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8836d5-ffb7-4466-9be6-6bab16b5326d service neutron] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:00.765641 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8836d5-ffb7-4466-9be6-6bab16b5326d service neutron] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:00.830498 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbc9115-5603-4d99-814d-70a83001571e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:00.831348 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbc9115-5603-4d99-814d-70a83001571e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:00.831631 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbc9115-5603-4d99-814d-70a83001571e admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:00.831907 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbc9115-5603-4d99-814d-70a83001571e admin admin] PATH_INFO: `/v3/projects/f4583d65a595463b97e8dfabbf3da521` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:00.833122 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbc9115-5603-4d99-814d-70a83001571e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4583d65a595463b97e8dfabbf3da521)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:00.835118 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbc9115-5603-4d99-814d-70a83001571e admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:00.881051 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:25:00.887913 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-6fbc9115-5603-4d99-814d-70a83001571e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4583d65a595463b97e8dfabbf3da521', 'request_id': 'req-6fbc9115-5603-4d99-814d-70a83001571e'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:00.895038 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-6fbc9115-5603-4d99-814d-70a83001571e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4583d65a595463b97e8dfabbf3da521', 'request_id': 'req-6fbc9115-5603-4d99-814d-70a83001571e'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:00.902856 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-6fbc9115-5603-4d99-814d-70a83001571e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4583d65a595463b97e8dfabbf3da521', 'request_id': 'req-6fbc9115-5603-4d99-814d-70a83001571e'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:00.915066 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-6fbc9115-5603-4d99-814d-70a83001571e admin admin] The token cache is being invalidate because project f4583d65a595463b97e8dfabbf3da521 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:25:00.915394 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-6fbc9115-5603-4d99-814d-70a83001571e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:00.916070 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-6fbc9115-5603-4d99-814d-70a83001571e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:00.917786 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1363/2740] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:25:00 2022] DELETE /identity/v3/projects/f4583d65a595463b97e8dfabbf3da521 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:25:00.923495 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623ed18a-46a2-4026-b23f-0304c51a9af7 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:00.923781 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623ed18a-46a2-4026-b23f-0304c51a9af7 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:00.924223 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623ed18a-46a2-4026-b23f-0304c51a9af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:00.971220 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-623ed18a-46a2-4026-b23f-0304c51a9af7 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:25:00.972415 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-623ed18a-46a2-4026-b23f-0304c51a9af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:01.032013 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1378/2741] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:25:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:25:01.040013 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a737d90-88c7-4022-804c-aee04c05193b None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:01.040307 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a737d90-88c7-4022-804c-aee04c05193b None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:01.040588 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a737d90-88c7-4022-804c-aee04c05193b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:01.060067 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-9a737d90-88c7-4022-804c-aee04c05193b None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:25:01.061653 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-9a737d90-88c7-4022-804c-aee04c05193b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:01.069359 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1364/2742] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:25:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:25:01.073444 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623ed18a-46a2-4026-b23f-0304c51a9af7 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:01.083712 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623ed18a-46a2-4026-b23f-0304c51a9af7 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:01.088203 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a496e3f-181c-41d3-bdb7-198440072dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:01.088781 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a496e3f-181c-41d3-bdb7-198440072dd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:01.089055 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a496e3f-181c-41d3-bdb7-198440072dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:01.089322 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a496e3f-181c-41d3-bdb7-198440072dd1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:01.089954 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a496e3f-181c-41d3-bdb7-198440072dd1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:25:01.090251 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a496e3f-181c-41d3-bdb7-198440072dd1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:01.091844 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a496e3f-181c-41d3-bdb7-198440072dd1 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:01.099306 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:25:01.106813 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1379/2743] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:25:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:25:01.111332 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a737d90-88c7-4022-804c-aee04c05193b None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:01.120459 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a737d90-88c7-4022-804c-aee04c05193b None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:01.125202 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35132754-d775-4521-a1e1-1ac78793acfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:01.125789 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35132754-d775-4521-a1e1-1ac78793acfa admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:01.126063 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35132754-d775-4521-a1e1-1ac78793acfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:01.126332 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35132754-d775-4521-a1e1-1ac78793acfa admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:01.126951 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35132754-d775-4521-a1e1-1ac78793acfa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:01.128869 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35132754-d775-4521-a1e1-1ac78793acfa admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:01.165858 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1365/2744] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:25:01 2022] POST /identity/v3/projects => generated 405 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:25:01.170173 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a496e3f-181c-41d3-bdb7-198440072dd1 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:01.180166 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a496e3f-181c-41d3-bdb7-198440072dd1 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:01.236410 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17082b54-0e13-4c03-9b2b-fec680ac93ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:01.237001 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17082b54-0e13-4c03-9b2b-fec680ac93ee admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:01.237277 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17082b54-0e13-4c03-9b2b-fec680ac93ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:01.237544 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17082b54-0e13-4c03-9b2b-fec680ac93ee admin admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:01.238164 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17082b54-0e13-4c03-9b2b-fec680ac93ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:01.240089 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17082b54-0e13-4c03-9b2b-fec680ac93ee admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:01.261319 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1380/2745] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:25:01 2022] POST /identity/v3/users => generated 303 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 12:25:01.266155 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35132754-d775-4521-a1e1-1ac78793acfa admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:01.276913 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35132754-d775-4521-a1e1-1ac78793acfa admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:01.281535 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eacdda-092e-4e78-8a59-123468c2a3c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:01.282174 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eacdda-092e-4e78-8a59-123468c2a3c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:01.282411 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eacdda-092e-4e78-8a59-123468c2a3c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:01.282665 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eacdda-092e-4e78-8a59-123468c2a3c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:01.283332 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eacdda-092e-4e78-8a59-123468c2a3c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:01.284915 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eacdda-092e-4e78-8a59-123468c2a3c9 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:01.294655 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1366/2746] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:25:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:25:01.298607 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17082b54-0e13-4c03-9b2b-fec680ac93ee admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:01.308224 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17082b54-0e13-4c03-9b2b-fec680ac93ee admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:01.313311 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9231234f-fdc1-485a-b3d9-f70cb32a8e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:01.313892 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9231234f-fdc1-485a-b3d9-f70cb32a8e7a admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:01.314167 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9231234f-fdc1-485a-b3d9-f70cb32a8e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:01.314434 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9231234f-fdc1-485a-b3d9-f70cb32a8e7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:01.315058 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9231234f-fdc1-485a-b3d9-f70cb32a8e7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:01.316654 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9231234f-fdc1-485a-b3d9-f70cb32a8e7a admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:01.327286 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1381/2747] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:25:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 12:25:01.331057 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eacdda-092e-4e78-8a59-123468c2a3c9 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:01.340681 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eacdda-092e-4e78-8a59-123468c2a3c9 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:01.345510 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e8337f-6504-434f-9b64-cdbe6ff4e9fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:01.346139 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e8337f-6504-434f-9b64-cdbe6ff4e9fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:01.346416 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e8337f-6504-434f-9b64-cdbe6ff4e9fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:01.346686 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e8337f-6504-434f-9b64-cdbe6ff4e9fc admin admin] PATH_INFO: `/v3/projects/74548b51adfe4a008562cce228599ec2/users/9114bd96d49144d5ab90bce575a98f12/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:01.361830 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e8337f-6504-434f-9b64-cdbe6ff4e9fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=74548b51adfe4a008562cce228599ec2, user_id=9114bd96d49144d5ab90bce575a98f12, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:01.364514 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e8337f-6504-434f-9b64-cdbe6ff4e9fc admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:01.377938 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1367/2748] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:25:01 2022] PUT /identity/v3/projects/74548b51adfe4a008562cce228599ec2/users/9114bd96d49144d5ab90bce575a98f12/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:25:01.382864 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dcd198-ac85-4a00-893b-6b7a8d0166ca None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:01.383155 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dcd198-ac85-4a00-893b-6b7a8d0166ca None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:01.383501 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dcd198-ac85-4a00-893b-6b7a8d0166ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:01.402327 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-f1dcd198-ac85-4a00-893b-6b7a8d0166ca None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:25:01.403551 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-f1dcd198-ac85-4a00-893b-6b7a8d0166ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:01.461539 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1382/2749] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:25:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:25:01.471548 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e8337f-6504-434f-9b64-cdbe6ff4e9fc admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:01.473375 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-c9e8337f-6504-434f-9b64-cdbe6ff4e9fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:01.529444 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e8337f-6504-434f-9b64-cdbe6ff4e9fc admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:01.533571 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990b6dfa-5d65-460b-923c-e73a901f5781 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:01.534108 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990b6dfa-5d65-460b-923c-e73a901f5781 service neutron] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:01.534382 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990b6dfa-5d65-460b-923c-e73a901f5781 service neutron] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:01.534649 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990b6dfa-5d65-460b-923c-e73a901f5781 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:01.543384 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990b6dfa-5d65-460b-923c-e73a901f5781 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:01.544806 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990b6dfa-5d65-460b-923c-e73a901f5781 service neutron] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:01.554505 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1368/2750] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:25:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:25:02.174424 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c5bf7-dc79-4f17-a6a6-cb6d7a6f579c None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:02.174710 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c5bf7-dc79-4f17-a6a6-cb6d7a6f579c None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:02.174987 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c5bf7-dc79-4f17-a6a6-cb6d7a6f579c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:02.204737 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-b34c5bf7-dc79-4f17-a6a6-cb6d7a6f579c None None] MFA Rules not processed for user `9114bd96d49144d5ab90bce575a98f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:25:02.205960 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-b34c5bf7-dc79-4f17-a6a6-cb6d7a6f579c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:02.272836 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1383/2751] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:25:02 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:25:02.277615 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990b6dfa-5d65-460b-923c-e73a901f5781 service neutron] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:02.287550 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990b6dfa-5d65-460b-923c-e73a901f5781 service neutron] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:02.292069 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3760894d-c765-4352-ae05-957afd93c17e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:02.292664 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3760894d-c765-4352-ae05-957afd93c17e admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:02.292945 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3760894d-c765-4352-ae05-957afd93c17e admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:02.293215 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3760894d-c765-4352-ae05-957afd93c17e admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:02.293881 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3760894d-c765-4352-ae05-957afd93c17e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:02.296412 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3760894d-c765-4352-ae05-957afd93c17e admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:02.328395 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1369/2752] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:25:02 2022] POST /identity/v3/projects => generated 403 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:25:02.329502 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34c5bf7-dc79-4f17-a6a6-cb6d7a6f579c None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:02.340507 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34c5bf7-dc79-4f17-a6a6-cb6d7a6f579c None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:02.344637 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:25:02.400833 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387e4f4b-1458-4e00-8816-3e6dd07796c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:02.401443 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387e4f4b-1458-4e00-8816-3e6dd07796c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:02.401713 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387e4f4b-1458-4e00-8816-3e6dd07796c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:02.401984 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387e4f4b-1458-4e00-8816-3e6dd07796c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:02.402605 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387e4f4b-1458-4e00-8816-3e6dd07796c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:02.404577 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387e4f4b-1458-4e00-8816-3e6dd07796c3 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:02.425352 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1384/2753] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:25:02 2022] POST /identity/v3/users => generated 308 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:25:02.429395 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3760894d-c765-4352-ae05-957afd93c17e admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:02.440254 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3760894d-c765-4352-ae05-957afd93c17e admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:02.444815 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbcb69a-6156-4963-947b-e7630b796f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:02.445405 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbcb69a-6156-4963-947b-e7630b796f9d admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:02.445682 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbcb69a-6156-4963-947b-e7630b796f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:02.445951 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbcb69a-6156-4963-947b-e7630b796f9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:02.446583 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbcb69a-6156-4963-947b-e7630b796f9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:02.448211 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbcb69a-6156-4963-947b-e7630b796f9d admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:02.458716 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1370/2754] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:25:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:25:02.462729 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387e4f4b-1458-4e00-8816-3e6dd07796c3 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:02.472579 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387e4f4b-1458-4e00-8816-3e6dd07796c3 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:02.477094 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b885fd1e-9cc0-4a4d-9221-7a1af718a673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:02.477726 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b885fd1e-9cc0-4a4d-9221-7a1af718a673 admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:02.478003 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b885fd1e-9cc0-4a4d-9221-7a1af718a673 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:02.478274 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b885fd1e-9cc0-4a4d-9221-7a1af718a673 admin admin] PATH_INFO: `/v3/projects/3ca10a69bc624794895bffb9fe44d39e/users/ef50044844ea40c5a0d2b06a784efb0a/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:02.493408 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b885fd1e-9cc0-4a4d-9221-7a1af718a673 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ca10a69bc624794895bffb9fe44d39e, user_id=ef50044844ea40c5a0d2b06a784efb0a, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:02.496142 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b885fd1e-9cc0-4a4d-9221-7a1af718a673 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:02.509739 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1385/2755] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:25:02 2022] PUT /identity/v3/projects/3ca10a69bc624794895bffb9fe44d39e/users/ef50044844ea40c5a0d2b06a784efb0a/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 12:25:02.513672 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbcb69a-6156-4963-947b-e7630b796f9d admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:02.524351 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbcb69a-6156-4963-947b-e7630b796f9d admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:02.550219 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:25:02.586225 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5412f5a4-5628-4eeb-b01a-f565699a72ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:02.586827 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5412f5a4-5628-4eeb-b01a-f565699a72ec admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:02.587111 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5412f5a4-5628-4eeb-b01a-f565699a72ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:02.587485 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5412f5a4-5628-4eeb-b01a-f565699a72ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:02.588118 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5412f5a4-5628-4eeb-b01a-f565699a72ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:02.589749 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5412f5a4-5628-4eeb-b01a-f565699a72ec admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:02.599590 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1371/2756] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:25:02 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:25:02.603754 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b885fd1e-9cc0-4a4d-9221-7a1af718a673 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:02.614819 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b885fd1e-9cc0-4a4d-9221-7a1af718a673 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:02.619496 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2014853-9d48-4b87-9c44-578ae8e2f9da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:02.620134 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2014853-9d48-4b87-9c44-578ae8e2f9da admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:02.620413 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2014853-9d48-4b87-9c44-578ae8e2f9da admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:02.620686 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2014853-9d48-4b87-9c44-578ae8e2f9da admin admin] PATH_INFO: `/v3/projects/3ca10a69bc624794895bffb9fe44d39e/users/ef50044844ea40c5a0d2b06a784efb0a/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:02.622589 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2014853-9d48-4b87-9c44-578ae8e2f9da admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ca10a69bc624794895bffb9fe44d39e, user_id=ef50044844ea40c5a0d2b06a784efb0a, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:02.625858 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2014853-9d48-4b87-9c44-578ae8e2f9da admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:02.635811 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1386/2757] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:25:02 2022] PUT /identity/v3/projects/3ca10a69bc624794895bffb9fe44d39e/users/ef50044844ea40c5a0d2b06a784efb0a/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:25:03.265153 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace5c2c5-05d8-4391-8b66-5812f7c940d6 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:03.265620 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace5c2c5-05d8-4391-8b66-5812f7c940d6 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:03.265898 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace5c2c5-05d8-4391-8b66-5812f7c940d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:03.296110 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-ace5c2c5-05d8-4391-8b66-5812f7c940d6 None None] MFA Rules not processed for user `ef50044844ea40c5a0d2b06a784efb0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:25:03.297340 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-ace5c2c5-05d8-4391-8b66-5812f7c940d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:03.368014 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1372/2758] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:25:03 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:25:04.377043 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2014853-9d48-4b87-9c44-578ae8e2f9da admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:04.440918 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2014853-9d48-4b87-9c44-578ae8e2f9da admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:04.445493 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1db6c3b-c6de-465c-b3b1-d51b2319acab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:04.446142 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1db6c3b-c6de-465c-b3b1-d51b2319acab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:04.446420 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1db6c3b-c6de-465c-b3b1-d51b2319acab admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:04.446718 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1db6c3b-c6de-465c-b3b1-d51b2319acab admin admin] PATH_INFO: `/v3/users/9114bd96d49144d5ab90bce575a98f12` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:04.447867 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1db6c3b-c6de-465c-b3b1-d51b2319acab admin admin] RBAC: Authorizing `identity:delete_user(user_id=9114bd96d49144d5ab90bce575a98f12)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:04.449792 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1db6c3b-c6de-465c-b3b1-d51b2319acab admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:04.494705 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-b1db6c3b-c6de-465c-b3b1-d51b2319acab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9114bd96d49144d5ab90bce575a98f12'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:04.501767 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-b1db6c3b-c6de-465c-b3b1-d51b2319acab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9114bd96d49144d5ab90bce575a98f12'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:04.502386 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-b1db6c3b-c6de-465c-b3b1-d51b2319acab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9114bd96d49144d5ab90bce575a98f12'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:04.507592 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=86,keystone:INSERT=6,keystone:DELETE=8 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:25:04.510115 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-b1db6c3b-c6de-465c-b3b1-d51b2319acab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9114bd96d49144d5ab90bce575a98f12'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:04.511315 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-b1db6c3b-c6de-465c-b3b1-d51b2319acab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9114bd96d49144d5ab90bce575a98f12'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:04.529152 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1387/2759] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:25:04 2022] DELETE /identity/v3/users/9114bd96d49144d5ab90bce575a98f12 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:25:04.697243 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace5c2c5-05d8-4391-8b66-5812f7c940d6 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:04.708961 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace5c2c5-05d8-4391-8b66-5812f7c940d6 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:04.770586 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6606a0a1-23c6-4b20-98a4-827a76b21885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:04.771246 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6606a0a1-23c6-4b20-98a4-827a76b21885 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:04.771533 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6606a0a1-23c6-4b20-98a4-827a76b21885 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:04.771806 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6606a0a1-23c6-4b20-98a4-827a76b21885 admin admin] PATH_INFO: `/v3/projects/74548b51adfe4a008562cce228599ec2` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:04.772946 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6606a0a1-23c6-4b20-98a4-827a76b21885 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74548b51adfe4a008562cce228599ec2)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:04.774877 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6606a0a1-23c6-4b20-98a4-827a76b21885 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:04.822825 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-6606a0a1-23c6-4b20-98a4-827a76b21885 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74548b51adfe4a008562cce228599ec2', 'request_id': 'req-6606a0a1-23c6-4b20-98a4-827a76b21885'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:04.823952 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:25:04.831569 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-6606a0a1-23c6-4b20-98a4-827a76b21885 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74548b51adfe4a008562cce228599ec2', 'request_id': 'req-6606a0a1-23c6-4b20-98a4-827a76b21885'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:04.838791 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-6606a0a1-23c6-4b20-98a4-827a76b21885 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74548b51adfe4a008562cce228599ec2', 'request_id': 'req-6606a0a1-23c6-4b20-98a4-827a76b21885'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:04.850405 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-6606a0a1-23c6-4b20-98a4-827a76b21885 admin admin] The token cache is being invalidate because project 74548b51adfe4a008562cce228599ec2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:25:04.850688 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-6606a0a1-23c6-4b20-98a4-827a76b21885 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:04.851342 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-6606a0a1-23c6-4b20-98a4-827a76b21885 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:04.852827 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1373/2760] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:25:04 2022] DELETE /identity/v3/projects/74548b51adfe4a008562cce228599ec2 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 12:25:04.856852 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1db6c3b-c6de-465c-b3b1-d51b2319acab admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:04.858408 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-b1db6c3b-c6de-465c-b3b1-d51b2319acab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:04.871590 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1db6c3b-c6de-465c-b3b1-d51b2319acab admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:04.926199 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f50a09-a8d7-4525-be36-9a71d1893709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:04.926847 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f50a09-a8d7-4525-be36-9a71d1893709 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:04.927121 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f50a09-a8d7-4525-be36-9a71d1893709 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:04.927421 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f50a09-a8d7-4525-be36-9a71d1893709 admin admin] PATH_INFO: `/v3/users/ef50044844ea40c5a0d2b06a784efb0a` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:04.928567 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f50a09-a8d7-4525-be36-9a71d1893709 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef50044844ea40c5a0d2b06a784efb0a)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:04.930750 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f50a09-a8d7-4525-be36-9a71d1893709 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:04.992352 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-21f50a09-a8d7-4525-be36-9a71d1893709 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef50044844ea40c5a0d2b06a784efb0a'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:05.001102 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-21f50a09-a8d7-4525-be36-9a71d1893709 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef50044844ea40c5a0d2b06a784efb0a'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:05.001715 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-21f50a09-a8d7-4525-be36-9a71d1893709 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef50044844ea40c5a0d2b06a784efb0a'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:05.009319 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-21f50a09-a8d7-4525-be36-9a71d1893709 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef50044844ea40c5a0d2b06a784efb0a'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:05.009952 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-21f50a09-a8d7-4525-be36-9a71d1893709 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef50044844ea40c5a0d2b06a784efb0a'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:05.027458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1388/2761] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:25:04 2022] DELETE /identity/v3/users/ef50044844ea40c5a0d2b06a784efb0a => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:25:05.200668 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6606a0a1-23c6-4b20-98a4-827a76b21885 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:05.202721 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-6606a0a1-23c6-4b20-98a4-827a76b21885 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:05.218525 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6606a0a1-23c6-4b20-98a4-827a76b21885 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:05.280741 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7b9681-93e0-4708-8f9f-4bebd303730d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:05.281286 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b9681-93e0-4708-8f9f-4bebd303730d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:05.281562 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b9681-93e0-4708-8f9f-4bebd303730d admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:05.281827 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b9681-93e0-4708-8f9f-4bebd303730d admin admin] PATH_INFO: `/v3/projects/3ca10a69bc624794895bffb9fe44d39e` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:05.282958 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7b9681-93e0-4708-8f9f-4bebd303730d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ca10a69bc624794895bffb9fe44d39e)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:05.284877 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7b9681-93e0-4708-8f9f-4bebd303730d admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:05.329783 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-de7b9681-93e0-4708-8f9f-4bebd303730d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ca10a69bc624794895bffb9fe44d39e', 'request_id': 'req-de7b9681-93e0-4708-8f9f-4bebd303730d'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:05.335692 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-de7b9681-93e0-4708-8f9f-4bebd303730d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ca10a69bc624794895bffb9fe44d39e', 'request_id': 'req-de7b9681-93e0-4708-8f9f-4bebd303730d'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:05.343699 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-de7b9681-93e0-4708-8f9f-4bebd303730d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ca10a69bc624794895bffb9fe44d39e', 'request_id': 'req-de7b9681-93e0-4708-8f9f-4bebd303730d'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:05.353715 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-de7b9681-93e0-4708-8f9f-4bebd303730d admin admin] The token cache is being invalidate because project 3ca10a69bc624794895bffb9fe44d39e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:25:05.354000 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-de7b9681-93e0-4708-8f9f-4bebd303730d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:05.354613 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-de7b9681-93e0-4708-8f9f-4bebd303730d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:05.356092 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1374/2762] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:25:05 2022] DELETE /identity/v3/projects/3ca10a69bc624794895bffb9fe44d39e => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:25:05.361605 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea11aec-74a6-4984-ae86-7fa1fdf12b7a None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:05.361917 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea11aec-74a6-4984-ae86-7fa1fdf12b7a None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:05.362191 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea11aec-74a6-4984-ae86-7fa1fdf12b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:05.381521 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-0ea11aec-74a6-4984-ae86-7fa1fdf12b7a None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:25:05.382705 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-0ea11aec-74a6-4984-ae86-7fa1fdf12b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:05.433508 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:25:05.444699 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1389/2763] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:25:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:25:05.452519 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517270c6-de45-4396-81f3-73addc04ae47 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:05.452800 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517270c6-de45-4396-81f3-73addc04ae47 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:05.453076 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517270c6-de45-4396-81f3-73addc04ae47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:05.473010 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-517270c6-de45-4396-81f3-73addc04ae47 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:25:05.474205 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-517270c6-de45-4396-81f3-73addc04ae47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:05.482136 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1375/2764] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:25:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:25:05.486592 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea11aec-74a6-4984-ae86-7fa1fdf12b7a None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:05.496082 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea11aec-74a6-4984-ae86-7fa1fdf12b7a None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:05.500707 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55e70a3-c0dc-4cb4-90c9-4bf1ffa5f92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:05.501299 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55e70a3-c0dc-4cb4-90c9-4bf1ffa5f92f admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:05.501577 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55e70a3-c0dc-4cb4-90c9-4bf1ffa5f92f admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:05.501845 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55e70a3-c0dc-4cb4-90c9-4bf1ffa5f92f admin admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:05.502482 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55e70a3-c0dc-4cb4-90c9-4bf1ffa5f92f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:25:05.502783 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55e70a3-c0dc-4cb4-90c9-4bf1ffa5f92f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:05.504396 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55e70a3-c0dc-4cb4-90c9-4bf1ffa5f92f admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:05.516456 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1390/2765] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:25:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:25:05.520563 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517270c6-de45-4396-81f3-73addc04ae47 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:05.530260 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517270c6-de45-4396-81f3-73addc04ae47 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:05.535157 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27ee67e-7d09-459e-bfa2-0dbbcdb12223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:05.535747 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ee67e-7d09-459e-bfa2-0dbbcdb12223 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:05.536029 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ee67e-7d09-459e-bfa2-0dbbcdb12223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:05.536301 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27ee67e-7d09-459e-bfa2-0dbbcdb12223 admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:05.536918 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27ee67e-7d09-459e-bfa2-0dbbcdb12223 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:05.538809 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27ee67e-7d09-459e-bfa2-0dbbcdb12223 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:05.566776 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1376/2766] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:25:05 2022] POST /identity/v3/projects => generated 365 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:25:05.570890 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55e70a3-c0dc-4cb4-90c9-4bf1ffa5f92f admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:05.580909 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55e70a3-c0dc-4cb4-90c9-4bf1ffa5f92f admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:05.635455 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb019fc-2624-413f-85ef-803f5e729c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:05.636042 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb019fc-2624-413f-85ef-803f5e729c07 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:05.636314 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb019fc-2624-413f-85ef-803f5e729c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:05.636586 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb019fc-2624-413f-85ef-803f5e729c07 admin admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:05.637204 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb019fc-2624-413f-85ef-803f5e729c07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:05.639192 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb019fc-2624-413f-85ef-803f5e729c07 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:05.659322 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1391/2767] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:25:05 2022] POST /identity/v3/users => generated 283 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 12:25:05.663392 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27ee67e-7d09-459e-bfa2-0dbbcdb12223 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:05.674159 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27ee67e-7d09-459e-bfa2-0dbbcdb12223 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:05.678702 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d3efa3-a73a-4813-9e93-a3aaafecaf83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:05.679318 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d3efa3-a73a-4813-9e93-a3aaafecaf83 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:05.679598 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d3efa3-a73a-4813-9e93-a3aaafecaf83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:05.679866 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d3efa3-a73a-4813-9e93-a3aaafecaf83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:05.680496 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d3efa3-a73a-4813-9e93-a3aaafecaf83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:05.682100 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d3efa3-a73a-4813-9e93-a3aaafecaf83 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:05.692438 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1377/2768] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:25:05.696090 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb019fc-2624-413f-85ef-803f5e729c07 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:05.705635 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb019fc-2624-413f-85ef-803f5e729c07 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:05.710150 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c2d7b5-1dbc-4fd6-995f-79ae8d20b23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:05.710728 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2d7b5-1dbc-4fd6-995f-79ae8d20b23a admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:05.711003 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2d7b5-1dbc-4fd6-995f-79ae8d20b23a admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:05.711305 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2d7b5-1dbc-4fd6-995f-79ae8d20b23a admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:05.711937 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c2d7b5-1dbc-4fd6-995f-79ae8d20b23a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:05.713507 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c2d7b5-1dbc-4fd6-995f-79ae8d20b23a admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:05.724300 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1392/2769] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:25:05.729636 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d3efa3-a73a-4813-9e93-a3aaafecaf83 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:05.740912 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d3efa3-a73a-4813-9e93-a3aaafecaf83 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:05.746534 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b1ee95-cc84-415d-bfc1-badc80d0dbbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:05.747188 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b1ee95-cc84-415d-bfc1-badc80d0dbbd admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:05.747499 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b1ee95-cc84-415d-bfc1-badc80d0dbbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:05.747774 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b1ee95-cc84-415d-bfc1-badc80d0dbbd admin admin] PATH_INFO: `/v3/projects/6a8789efed7d48a6bb9fa6783382df3b/users/3ad338368b8a465e9022d216a605a36c/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:05.762539 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b1ee95-cc84-415d-bfc1-badc80d0dbbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a8789efed7d48a6bb9fa6783382df3b, user_id=3ad338368b8a465e9022d216a605a36c, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:05.765234 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b1ee95-cc84-415d-bfc1-badc80d0dbbd admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:05.777723 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1378/2770] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:25:05 2022] PUT /identity/v3/projects/6a8789efed7d48a6bb9fa6783382df3b/users/3ad338368b8a465e9022d216a605a36c/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:25:05.790117 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71440574-509a-4657-8eec-b81f352e2c08 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:05.790320 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71440574-509a-4657-8eec-b81f352e2c08 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:05.790596 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71440574-509a-4657-8eec-b81f352e2c08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:05.817845 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-71440574-509a-4657-8eec-b81f352e2c08 None None] MFA Rules not processed for user `3ad338368b8a465e9022d216a605a36c`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:25:05.819038 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-71440574-509a-4657-8eec-b81f352e2c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:05.881816 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1393/2771] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:25:05 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:25:05.886207 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b1ee95-cc84-415d-bfc1-badc80d0dbbd admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:05.891961 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:25:05.951080 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b1ee95-cc84-415d-bfc1-badc80d0dbbd admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:05.955805 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97dc2f8-51d9-49ab-8bff-4fcc205afc55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:05.956425 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97dc2f8-51d9-49ab-8bff-4fcc205afc55 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:05.956709 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97dc2f8-51d9-49ab-8bff-4fcc205afc55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:05.957004 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97dc2f8-51d9-49ab-8bff-4fcc205afc55 admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:05.957643 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97dc2f8-51d9-49ab-8bff-4fcc205afc55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:05.959554 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97dc2f8-51d9-49ab-8bff-4fcc205afc55 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:05.987716 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1379/2772] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:25:05 2022] POST /identity/v3/projects => generated 365 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:25:05.991800 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71440574-509a-4657-8eec-b81f352e2c08 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:06.002719 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71440574-509a-4657-8eec-b81f352e2c08 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:06.050618 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:25:06.060313 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5698e8a-b462-40c4-bd91-734f26e4b568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:06.060949 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5698e8a-b462-40c4-bd91-734f26e4b568 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:06.061231 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5698e8a-b462-40c4-bd91-734f26e4b568 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:06.061595 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5698e8a-b462-40c4-bd91-734f26e4b568 admin admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:06.062217 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5698e8a-b462-40c4-bd91-734f26e4b568 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:06.064207 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5698e8a-b462-40c4-bd91-734f26e4b568 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:06.085616 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1394/2773] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:25:05 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:25:06.089602 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97dc2f8-51d9-49ab-8bff-4fcc205afc55 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:06.100483 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97dc2f8-51d9-49ab-8bff-4fcc205afc55 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:06.105213 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b4c77f-c45b-4021-ad41-65e88b01b4a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:06.105813 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b4c77f-c45b-4021-ad41-65e88b01b4a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:06.106092 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b4c77f-c45b-4021-ad41-65e88b01b4a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:06.106365 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b4c77f-c45b-4021-ad41-65e88b01b4a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:06.107035 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b4c77f-c45b-4021-ad41-65e88b01b4a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:06.108667 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b4c77f-c45b-4021-ad41-65e88b01b4a5 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:06.119530 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1380/2774] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:25:06.123442 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5698e8a-b462-40c4-bd91-734f26e4b568 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:06.133223 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5698e8a-b462-40c4-bd91-734f26e4b568 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:06.137776 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6fe75c-f64e-45c2-9bd8-c50b8cdf70c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:06.138411 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6fe75c-f64e-45c2-9bd8-c50b8cdf70c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:06.138688 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6fe75c-f64e-45c2-9bd8-c50b8cdf70c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:06.139622 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6fe75c-f64e-45c2-9bd8-c50b8cdf70c5 admin admin] PATH_INFO: `/v3/projects/62c4887205f84219a01c7c9f0b47b0c5/users/dc07c46458ae4f8b8320bc4dd3d090c8/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:06.155574 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6fe75c-f64e-45c2-9bd8-c50b8cdf70c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62c4887205f84219a01c7c9f0b47b0c5, user_id=dc07c46458ae4f8b8320bc4dd3d090c8, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:06.158186 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6fe75c-f64e-45c2-9bd8-c50b8cdf70c5 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:06.171038 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1395/2775] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:25:06 2022] PUT /identity/v3/projects/62c4887205f84219a01c7c9f0b47b0c5/users/dc07c46458ae4f8b8320bc4dd3d090c8/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:25:06.174910 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b4c77f-c45b-4021-ad41-65e88b01b4a5 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:06.184844 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b4c77f-c45b-4021-ad41-65e88b01b4a5 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:06.240057 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13460da4-9016-4dd2-b637-3bddf375793d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:06.240653 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13460da4-9016-4dd2-b637-3bddf375793d admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:06.240933 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13460da4-9016-4dd2-b637-3bddf375793d admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:06.241209 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13460da4-9016-4dd2-b637-3bddf375793d admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:06.241836 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13460da4-9016-4dd2-b637-3bddf375793d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:06.243450 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13460da4-9016-4dd2-b637-3bddf375793d admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:06.253235 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1381/2776] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:25:06.257304 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6fe75c-f64e-45c2-9bd8-c50b8cdf70c5 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:06.268196 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6fe75c-f64e-45c2-9bd8-c50b8cdf70c5 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:06.272773 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28f04bc-5263-4244-80ab-580fffc78e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:06.273533 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f04bc-5263-4244-80ab-580fffc78e88 admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:06.273932 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f04bc-5263-4244-80ab-580fffc78e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:06.274248 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f04bc-5263-4244-80ab-580fffc78e88 admin admin] PATH_INFO: `/v3/projects/62c4887205f84219a01c7c9f0b47b0c5/users/dc07c46458ae4f8b8320bc4dd3d090c8/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:06.276700 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28f04bc-5263-4244-80ab-580fffc78e88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62c4887205f84219a01c7c9f0b47b0c5, user_id=dc07c46458ae4f8b8320bc4dd3d090c8, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:06.279317 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28f04bc-5263-4244-80ab-580fffc78e88 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:06.289747 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1396/2777] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:25:06 2022] PUT /identity/v3/projects/62c4887205f84219a01c7c9f0b47b0c5/users/dc07c46458ae4f8b8320bc4dd3d090c8/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:25:06.300030 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569843e6-713d-4153-8119-70912ce59415 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:06.300318 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569843e6-713d-4153-8119-70912ce59415 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:06.300593 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569843e6-713d-4153-8119-70912ce59415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:06.329024 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-569843e6-713d-4153-8119-70912ce59415 None None] MFA Rules not processed for user `dc07c46458ae4f8b8320bc4dd3d090c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:25:06.330137 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-569843e6-713d-4153-8119-70912ce59415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:06.517636 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:25:06.521606 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1382/2778] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:25:06 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:25:06.533025 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28f04bc-5263-4244-80ab-580fffc78e88 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:06.534490 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-a28f04bc-5263-4244-80ab-580fffc78e88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:06.611815 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28f04bc-5263-4244-80ab-580fffc78e88 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:06.615357 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1f5667-b6ff-4b88-9e59-b6226f35dc9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:06.615943 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f5667-b6ff-4b88-9e59-b6226f35dc9d service cinder] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:06.616219 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f5667-b6ff-4b88-9e59-b6226f35dc9d service cinder] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:06.616487 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f5667-b6ff-4b88-9e59-b6226f35dc9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:06.625945 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1f5667-b6ff-4b88-9e59-b6226f35dc9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:06.627376 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1f5667-b6ff-4b88-9e59-b6226f35dc9d service cinder] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:06.636454 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1397/2779] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:25:06 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:25:06.711111 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569843e6-713d-4153-8119-70912ce59415 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:06.720693 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569843e6-713d-4153-8119-70912ce59415 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:06.724269 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c918cb49-9840-4267-8ca4-3d982ed45e88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:06.724860 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c918cb49-9840-4267-8ca4-3d982ed45e88 service cinder] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:06.725131 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c918cb49-9840-4267-8ca4-3d982ed45e88 service cinder] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:06.725397 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c918cb49-9840-4267-8ca4-3d982ed45e88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:06.782486 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c918cb49-9840-4267-8ca4-3d982ed45e88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:06.783929 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c918cb49-9840-4267-8ca4-3d982ed45e88 service cinder] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:06.792812 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1383/2780] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:25:06 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:25:16.072821 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1f5667-b6ff-4b88-9e59-b6226f35dc9d service cinder] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:16.134460 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1f5667-b6ff-4b88-9e59-b6226f35dc9d service cinder] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:16.139664 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7970efa9-d5db-4e1d-bfd5-6266577ac370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:16.140281 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7970efa9-d5db-4e1d-bfd5-6266577ac370 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:16.140559 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7970efa9-d5db-4e1d-bfd5-6266577ac370 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:16.140831 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7970efa9-d5db-4e1d-bfd5-6266577ac370 admin admin] PATH_INFO: `/v3/users/3ad338368b8a465e9022d216a605a36c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:16.141929 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7970efa9-d5db-4e1d-bfd5-6266577ac370 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ad338368b8a465e9022d216a605a36c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:16.143871 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7970efa9-d5db-4e1d-bfd5-6266577ac370 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:16.175718 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:25:16.191656 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-7970efa9-d5db-4e1d-bfd5-6266577ac370 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ad338368b8a465e9022d216a605a36c'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:16.199108 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-7970efa9-d5db-4e1d-bfd5-6266577ac370 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ad338368b8a465e9022d216a605a36c'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:16.199775 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-7970efa9-d5db-4e1d-bfd5-6266577ac370 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ad338368b8a465e9022d216a605a36c'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:16.207450 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-7970efa9-d5db-4e1d-bfd5-6266577ac370 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ad338368b8a465e9022d216a605a36c'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:16.208086 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-7970efa9-d5db-4e1d-bfd5-6266577ac370 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ad338368b8a465e9022d216a605a36c'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:16.224881 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1398/2781] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:25:16 2022] DELETE /identity/v3/users/3ad338368b8a465e9022d216a605a36c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:25:16.229409 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e82fc52-6a20-4812-8405-ba1c7c9c95b8 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:16.229878 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e82fc52-6a20-4812-8405-ba1c7c9c95b8 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:16.229994 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e82fc52-6a20-4812-8405-ba1c7c9c95b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:16.248866 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-3e82fc52-6a20-4812-8405-ba1c7c9c95b8 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:25:16.250138 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-3e82fc52-6a20-4812-8405-ba1c7c9c95b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:16.307910 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1384/2782] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:25:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:25:16.318452 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7970efa9-d5db-4e1d-bfd5-6266577ac370 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:16.319946 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-7970efa9-d5db-4e1d-bfd5-6266577ac370 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:16.376889 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7970efa9-d5db-4e1d-bfd5-6266577ac370 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:16.380493 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824a9856-4287-4f8b-9b05-600db248df36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:16.381085 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824a9856-4287-4f8b-9b05-600db248df36 service neutron] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:16.381360 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824a9856-4287-4f8b-9b05-600db248df36 service neutron] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:16.381626 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824a9856-4287-4f8b-9b05-600db248df36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:16.391167 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824a9856-4287-4f8b-9b05-600db248df36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:16.392610 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824a9856-4287-4f8b-9b05-600db248df36 service neutron] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:16.401876 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1399/2783] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:25:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:25:16.665084 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e82fc52-6a20-4812-8405-ba1c7c9c95b8 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:16.666526 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-3e82fc52-6a20-4812-8405-ba1c7c9c95b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:16.679642 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e82fc52-6a20-4812-8405-ba1c7c9c95b8 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:16.684240 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7365aa-4ea1-4ff1-ac3a-09686ef2d0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:16.685010 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7365aa-4ea1-4ff1-ac3a-09686ef2d0e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:16.685293 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7365aa-4ea1-4ff1-ac3a-09686ef2d0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:16.686116 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7365aa-4ea1-4ff1-ac3a-09686ef2d0e8 admin admin] PATH_INFO: `/v3/projects/6a8789efed7d48a6bb9fa6783382df3b` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:16.687281 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7365aa-4ea1-4ff1-ac3a-09686ef2d0e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a8789efed7d48a6bb9fa6783382df3b)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:16.689185 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7365aa-4ea1-4ff1-ac3a-09686ef2d0e8 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:16.736785 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-aa7365aa-4ea1-4ff1-ac3a-09686ef2d0e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a8789efed7d48a6bb9fa6783382df3b', 'request_id': 'req-aa7365aa-4ea1-4ff1-ac3a-09686ef2d0e8'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:16.743128 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-aa7365aa-4ea1-4ff1-ac3a-09686ef2d0e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a8789efed7d48a6bb9fa6783382df3b', 'request_id': 'req-aa7365aa-4ea1-4ff1-ac3a-09686ef2d0e8'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:16.754787 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-aa7365aa-4ea1-4ff1-ac3a-09686ef2d0e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a8789efed7d48a6bb9fa6783382df3b', 'request_id': 'req-aa7365aa-4ea1-4ff1-ac3a-09686ef2d0e8'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:16.765416 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-aa7365aa-4ea1-4ff1-ac3a-09686ef2d0e8 admin admin] The token cache is being invalidate because project 6a8789efed7d48a6bb9fa6783382df3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:25:16.765708 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-aa7365aa-4ea1-4ff1-ac3a-09686ef2d0e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:16.766355 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-aa7365aa-4ea1-4ff1-ac3a-09686ef2d0e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:16.767879 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1385/2784] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:25:16 2022] DELETE /identity/v3/projects/6a8789efed7d48a6bb9fa6783382df3b => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:25:16.771869 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824a9856-4287-4f8b-9b05-600db248df36 service neutron] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:16.782027 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824a9856-4287-4f8b-9b05-600db248df36 service neutron] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:16.838460 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d079a9c7-666c-4c9f-9bb4-d4c7fac7d166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:16.839000 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079a9c7-666c-4c9f-9bb4-d4c7fac7d166 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:16.839314 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079a9c7-666c-4c9f-9bb4-d4c7fac7d166 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:16.839589 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079a9c7-666c-4c9f-9bb4-d4c7fac7d166 admin admin] PATH_INFO: `/v3/users/dc07c46458ae4f8b8320bc4dd3d090c8` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:16.840693 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d079a9c7-666c-4c9f-9bb4-d4c7fac7d166 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc07c46458ae4f8b8320bc4dd3d090c8)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:16.842835 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d079a9c7-666c-4c9f-9bb4-d4c7fac7d166 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:16.888679 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-d079a9c7-666c-4c9f-9bb4-d4c7fac7d166 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc07c46458ae4f8b8320bc4dd3d090c8'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:16.892283 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:25:16.896518 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-d079a9c7-666c-4c9f-9bb4-d4c7fac7d166 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc07c46458ae4f8b8320bc4dd3d090c8'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:16.897292 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-d079a9c7-666c-4c9f-9bb4-d4c7fac7d166 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc07c46458ae4f8b8320bc4dd3d090c8'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:16.905215 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-d079a9c7-666c-4c9f-9bb4-d4c7fac7d166 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc07c46458ae4f8b8320bc4dd3d090c8'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:16.905845 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-d079a9c7-666c-4c9f-9bb4-d4c7fac7d166 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc07c46458ae4f8b8320bc4dd3d090c8'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:16.923619 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1400/2785] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:25:16 2022] DELETE /identity/v3/users/dc07c46458ae4f8b8320bc4dd3d090c8 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:25:17.189275 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7365aa-4ea1-4ff1-ac3a-09686ef2d0e8 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:17.190765 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-aa7365aa-4ea1-4ff1-ac3a-09686ef2d0e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:17.206744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7365aa-4ea1-4ff1-ac3a-09686ef2d0e8 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:17.227776 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:25:17.268904 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce90c7b-28a2-4e5d-bb73-0e0dc862b421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:17.269566 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce90c7b-28a2-4e5d-bb73-0e0dc862b421 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:17.269800 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce90c7b-28a2-4e5d-bb73-0e0dc862b421 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:17.270069 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce90c7b-28a2-4e5d-bb73-0e0dc862b421 admin admin] PATH_INFO: `/v3/projects/62c4887205f84219a01c7c9f0b47b0c5` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:17.271324 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce90c7b-28a2-4e5d-bb73-0e0dc862b421 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62c4887205f84219a01c7c9f0b47b0c5)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:17.273270 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce90c7b-28a2-4e5d-bb73-0e0dc862b421 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:17.320863 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-2ce90c7b-28a2-4e5d-bb73-0e0dc862b421 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62c4887205f84219a01c7c9f0b47b0c5', 'request_id': 'req-2ce90c7b-28a2-4e5d-bb73-0e0dc862b421'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:17.328041 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-2ce90c7b-28a2-4e5d-bb73-0e0dc862b421 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62c4887205f84219a01c7c9f0b47b0c5', 'request_id': 'req-2ce90c7b-28a2-4e5d-bb73-0e0dc862b421'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:17.336042 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-2ce90c7b-28a2-4e5d-bb73-0e0dc862b421 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62c4887205f84219a01c7c9f0b47b0c5', 'request_id': 'req-2ce90c7b-28a2-4e5d-bb73-0e0dc862b421'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:17.346765 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-2ce90c7b-28a2-4e5d-bb73-0e0dc862b421 admin admin] The token cache is being invalidate because project 62c4887205f84219a01c7c9f0b47b0c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:25:17.347050 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-2ce90c7b-28a2-4e5d-bb73-0e0dc862b421 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:17.347720 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-2ce90c7b-28a2-4e5d-bb73-0e0dc862b421 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:17.349173 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1386/2786] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:25:17 2022] DELETE /identity/v3/projects/62c4887205f84219a01c7c9f0b47b0c5 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:25:17.354007 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9bd6cc-08c7-4ac2-a6c2-274aa000399c None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:17.354482 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9bd6cc-08c7-4ac2-a6c2-274aa000399c None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:17.354579 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9bd6cc-08c7-4ac2-a6c2-274aa000399c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:17.373925 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-3b9bd6cc-08c7-4ac2-a6c2-274aa000399c None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:25:17.375038 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-3b9bd6cc-08c7-4ac2-a6c2-274aa000399c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:17.433342 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1401/2787] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:25:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:25:17.441333 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5506b-c427-473f-9f0c-6910f8b9f998 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:17.441650 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5506b-c427-473f-9f0c-6910f8b9f998 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:17.441954 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5506b-c427-473f-9f0c-6910f8b9f998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:17.461554 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-31d5506b-c427-473f-9f0c-6910f8b9f998 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:25:17.462749 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-31d5506b-c427-473f-9f0c-6910f8b9f998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:17.471150 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1387/2788] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:25:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:25:17.475606 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9bd6cc-08c7-4ac2-a6c2-274aa000399c None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:17.486137 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9bd6cc-08c7-4ac2-a6c2-274aa000399c None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:17.490710 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b74c85-5a25-42df-8c69-8d6f8b41b0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:17.491337 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b74c85-5a25-42df-8c69-8d6f8b41b0f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:17.491620 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b74c85-5a25-42df-8c69-8d6f8b41b0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:17.491893 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b74c85-5a25-42df-8c69-8d6f8b41b0f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:17.492532 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b74c85-5a25-42df-8c69-8d6f8b41b0f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:25:17.492833 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b74c85-5a25-42df-8c69-8d6f8b41b0f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:17.494410 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b74c85-5a25-42df-8c69-8d6f8b41b0f5 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:17.506486 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1402/2789] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:25:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:25:17.510721 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d5506b-c427-473f-9f0c-6910f8b9f998 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:17.520615 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d5506b-c427-473f-9f0c-6910f8b9f998 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:17.525645 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5873ecd9-2774-4c35-9611-c40336933a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:17.526235 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5873ecd9-2774-4c35-9611-c40336933a62 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:17.526515 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5873ecd9-2774-4c35-9611-c40336933a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:17.526786 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5873ecd9-2774-4c35-9611-c40336933a62 admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:17.527439 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5873ecd9-2774-4c35-9611-c40336933a62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:17.529400 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5873ecd9-2774-4c35-9611-c40336933a62 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:17.556429 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1388/2790] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:25:17 2022] POST /identity/v3/projects => generated 367 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:25:17.560501 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b74c85-5a25-42df-8c69-8d6f8b41b0f5 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:17.570827 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b74c85-5a25-42df-8c69-8d6f8b41b0f5 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:17.626821 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77835fbf-5c8b-4129-9106-7f3850104065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:17.627390 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77835fbf-5c8b-4129-9106-7f3850104065 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:17.627666 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77835fbf-5c8b-4129-9106-7f3850104065 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:17.627937 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77835fbf-5c8b-4129-9106-7f3850104065 admin admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:17.628562 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77835fbf-5c8b-4129-9106-7f3850104065 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:17.630490 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77835fbf-5c8b-4129-9106-7f3850104065 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:17.650733 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1403/2791] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:25:17 2022] POST /identity/v3/users => generated 284 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 12:25:17.654817 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5873ecd9-2774-4c35-9611-c40336933a62 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:17.664796 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5873ecd9-2774-4c35-9611-c40336933a62 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:17.669416 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d356a63-2232-4c15-b392-d0ff65dbd95f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:17.670524 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d356a63-2232-4c15-b392-d0ff65dbd95f admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:17.670815 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d356a63-2232-4c15-b392-d0ff65dbd95f admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:17.671082 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d356a63-2232-4c15-b392-d0ff65dbd95f admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:17.671738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d356a63-2232-4c15-b392-d0ff65dbd95f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:17.673378 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d356a63-2232-4c15-b392-d0ff65dbd95f admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:17.683247 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1389/2792] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:25:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:25:17.687640 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77835fbf-5c8b-4129-9106-7f3850104065 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:17.698199 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77835fbf-5c8b-4129-9106-7f3850104065 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:17.703422 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5fd21f-feba-48e6-ac4a-00021fb82477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:17.709646 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5fd21f-feba-48e6-ac4a-00021fb82477 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:17.709646 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5fd21f-feba-48e6-ac4a-00021fb82477 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:17.709646 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5fd21f-feba-48e6-ac4a-00021fb82477 admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:17.709646 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5fd21f-feba-48e6-ac4a-00021fb82477 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:17.709646 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5fd21f-feba-48e6-ac4a-00021fb82477 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:17.718157 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1404/2793] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:25:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:25:17.722400 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d356a63-2232-4c15-b392-d0ff65dbd95f admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:17.733195 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d356a63-2232-4c15-b392-d0ff65dbd95f admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:17.738043 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5d5b7c-d72d-497a-bae9-0135ff8bd465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:17.738677 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d5b7c-d72d-497a-bae9-0135ff8bd465 admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:17.738957 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d5b7c-d72d-497a-bae9-0135ff8bd465 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:17.739254 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d5b7c-d72d-497a-bae9-0135ff8bd465 admin admin] PATH_INFO: `/v3/projects/20dfbfbb3c6c4bada2aa77d3c8a76630/users/ceb01168046243adbf630844c42b4106/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:17.755320 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5d5b7c-d72d-497a-bae9-0135ff8bd465 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20dfbfbb3c6c4bada2aa77d3c8a76630, user_id=ceb01168046243adbf630844c42b4106, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:17.757953 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5d5b7c-d72d-497a-bae9-0135ff8bd465 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:17.771811 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1390/2794] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:25:17 2022] PUT /identity/v3/projects/20dfbfbb3c6c4bada2aa77d3c8a76630/users/ceb01168046243adbf630844c42b4106/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:25:17.836459 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df54448-5328-4f30-b356-d32ac276638d None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:17.836760 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df54448-5328-4f30-b356-d32ac276638d None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:17.837036 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df54448-5328-4f30-b356-d32ac276638d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:17.866146 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-4df54448-5328-4f30-b356-d32ac276638d None None] MFA Rules not processed for user `ceb01168046243adbf630844c42b4106`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:25:17.867379 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-4df54448-5328-4f30-b356-d32ac276638d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:17.890374 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:25:17.930529 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1405/2795] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:25:17 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:25:17.935191 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5d5b7c-d72d-497a-bae9-0135ff8bd465 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:17.982216 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:25:17.997531 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5d5b7c-d72d-497a-bae9-0135ff8bd465 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:18.002346 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f1700d-aba7-4dff-9f10-2280bb77fd2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:18.002942 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f1700d-aba7-4dff-9f10-2280bb77fd2e admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:18.003245 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f1700d-aba7-4dff-9f10-2280bb77fd2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:18.003524 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f1700d-aba7-4dff-9f10-2280bb77fd2e admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:18.004184 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f1700d-aba7-4dff-9f10-2280bb77fd2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:18.006061 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f1700d-aba7-4dff-9f10-2280bb77fd2e admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:18.031998 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1391/2796] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:25:17 2022] POST /identity/v3/projects => generated 369 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:25:18.036143 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df54448-5328-4f30-b356-d32ac276638d None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:18.047816 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df54448-5328-4f30-b356-d32ac276638d None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:18.104481 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb234fb-da14-4b4b-ade7-53c3bbbf9024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:18.105084 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb234fb-da14-4b4b-ade7-53c3bbbf9024 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:18.105364 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb234fb-da14-4b4b-ade7-53c3bbbf9024 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:18.105636 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb234fb-da14-4b4b-ade7-53c3bbbf9024 admin admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:18.106265 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb234fb-da14-4b4b-ade7-53c3bbbf9024 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:18.108334 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb234fb-da14-4b4b-ade7-53c3bbbf9024 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:18.129536 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1406/2797] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:25:18 2022] POST /identity/v3/users => generated 291 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 12:25:18.134058 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f1700d-aba7-4dff-9f10-2280bb77fd2e admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:18.145043 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f1700d-aba7-4dff-9f10-2280bb77fd2e admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:18.149552 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae412573-56cd-4e13-a5a1-2ea811bcff32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:18.150145 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae412573-56cd-4e13-a5a1-2ea811bcff32 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:18.150424 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae412573-56cd-4e13-a5a1-2ea811bcff32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:18.150694 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae412573-56cd-4e13-a5a1-2ea811bcff32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:18.151360 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae412573-56cd-4e13-a5a1-2ea811bcff32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:18.152961 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae412573-56cd-4e13-a5a1-2ea811bcff32 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:18.163962 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1392/2798] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:25:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 12:25:18.168173 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb234fb-da14-4b4b-ade7-53c3bbbf9024 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:18.178190 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb234fb-da14-4b4b-ade7-53c3bbbf9024 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:18.182811 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec9c4f0-ea74-4644-9941-c922df0c0426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:18.183477 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9c4f0-ea74-4644-9941-c922df0c0426 admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:18.183756 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9c4f0-ea74-4644-9941-c922df0c0426 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:18.184027 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9c4f0-ea74-4644-9941-c922df0c0426 admin admin] PATH_INFO: `/v3/projects/ec0e812ebda84f8b83ab51922cefdbd7/users/1a2496c48ef446e2823cea9ee043c283/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:18.198810 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec9c4f0-ea74-4644-9941-c922df0c0426 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec0e812ebda84f8b83ab51922cefdbd7, user_id=1a2496c48ef446e2823cea9ee043c283, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:18.202024 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec9c4f0-ea74-4644-9941-c922df0c0426 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:18.215370 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1407/2799] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:25:18 2022] PUT /identity/v3/projects/ec0e812ebda84f8b83ab51922cefdbd7/users/1a2496c48ef446e2823cea9ee043c283/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:25:18.219835 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae412573-56cd-4e13-a5a1-2ea811bcff32 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:18.229478 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae412573-56cd-4e13-a5a1-2ea811bcff32 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:18.287268 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91b6b30-4a48-4d3f-9d5f-619056304d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:18.287895 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b6b30-4a48-4d3f-9d5f-619056304d3b admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:18.288135 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b6b30-4a48-4d3f-9d5f-619056304d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:18.288399 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b6b30-4a48-4d3f-9d5f-619056304d3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:18.289023 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91b6b30-4a48-4d3f-9d5f-619056304d3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:18.290592 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91b6b30-4a48-4d3f-9d5f-619056304d3b admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:18.300744 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1393/2800] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:25:18 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:25:18.304644 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec9c4f0-ea74-4644-9941-c922df0c0426 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:18.314744 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec9c4f0-ea74-4644-9941-c922df0c0426 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:18.319298 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09c9e7f-15d2-403d-b8e9-47308ce786b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:18.319929 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c9e7f-15d2-403d-b8e9-47308ce786b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:18.320202 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c9e7f-15d2-403d-b8e9-47308ce786b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:18.320470 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c9e7f-15d2-403d-b8e9-47308ce786b5 admin admin] PATH_INFO: `/v3/projects/ec0e812ebda84f8b83ab51922cefdbd7/users/1a2496c48ef446e2823cea9ee043c283/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:18.322342 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09c9e7f-15d2-403d-b8e9-47308ce786b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec0e812ebda84f8b83ab51922cefdbd7, user_id=1a2496c48ef446e2823cea9ee043c283, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:18.325028 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09c9e7f-15d2-403d-b8e9-47308ce786b5 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:18.335453 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1408/2801] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:25:18 2022] PUT /identity/v3/projects/ec0e812ebda84f8b83ab51922cefdbd7/users/1a2496c48ef446e2823cea9ee043c283/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:25:18.345978 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b5aa8-0265-42d1-8a1c-734932eab802 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:18.346289 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b5aa8-0265-42d1-8a1c-734932eab802 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:18.346563 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b5aa8-0265-42d1-8a1c-734932eab802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:18.373454 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-ee0b5aa8-0265-42d1-8a1c-734932eab802 None None] MFA Rules not processed for user `1a2496c48ef446e2823cea9ee043c283`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:25:18.374748 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-ee0b5aa8-0265-42d1-8a1c-734932eab802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:18.440338 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1394/2802] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:25:18 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:25:18.452124 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09c9e7f-15d2-403d-b8e9-47308ce786b5 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:18.453836 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-d09c9e7f-15d2-403d-b8e9-47308ce786b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:18.509951 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09c9e7f-15d2-403d-b8e9-47308ce786b5 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:18.514337 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7fec8c-45b0-46d7-a56e-2268d669cad1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:18.514926 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7fec8c-45b0-46d7-a56e-2268d669cad1 service cinder] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:18.515218 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7fec8c-45b0-46d7-a56e-2268d669cad1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:18.515502 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7fec8c-45b0-46d7-a56e-2268d669cad1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:18.524245 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7fec8c-45b0-46d7-a56e-2268d669cad1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:18.525658 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7fec8c-45b0-46d7-a56e-2268d669cad1 service cinder] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:18.534880 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1409/2803] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:25:18 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:25:19.869067 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0b5aa8-0265-42d1-8a1c-734932eab802 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:19.914378 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:25:19.931737 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0b5aa8-0265-42d1-8a1c-734932eab802 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:19.936720 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b049fb8b-6ad5-44a3-8832-1ea950fe6bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:19.937338 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b049fb8b-6ad5-44a3-8832-1ea950fe6bc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:19.937620 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b049fb8b-6ad5-44a3-8832-1ea950fe6bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:19.937895 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b049fb8b-6ad5-44a3-8832-1ea950fe6bc9 admin admin] PATH_INFO: `/v3/users/ceb01168046243adbf630844c42b4106` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:19.939000 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b049fb8b-6ad5-44a3-8832-1ea950fe6bc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ceb01168046243adbf630844c42b4106)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:19.940951 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b049fb8b-6ad5-44a3-8832-1ea950fe6bc9 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:19.989009 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-b049fb8b-6ad5-44a3-8832-1ea950fe6bc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ceb01168046243adbf630844c42b4106'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:19.996125 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-b049fb8b-6ad5-44a3-8832-1ea950fe6bc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ceb01168046243adbf630844c42b4106'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:20.003618 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-b049fb8b-6ad5-44a3-8832-1ea950fe6bc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ceb01168046243adbf630844c42b4106'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:20.004276 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-b049fb8b-6ad5-44a3-8832-1ea950fe6bc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ceb01168046243adbf630844c42b4106'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:20.016435 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-b049fb8b-6ad5-44a3-8832-1ea950fe6bc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ceb01168046243adbf630844c42b4106'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:20.022240 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1395/2804] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:25:19 2022] DELETE /identity/v3/users/ceb01168046243adbf630844c42b4106 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:25:20.026761 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023c4f7f-cc2c-4e10-a0dd-a4e9f4e58941 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:20.027269 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023c4f7f-cc2c-4e10-a0dd-a4e9f4e58941 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:20.027423 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023c4f7f-cc2c-4e10-a0dd-a4e9f4e58941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:20.045617 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-023c4f7f-cc2c-4e10-a0dd-a4e9f4e58941 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:25:20.046795 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-023c4f7f-cc2c-4e10-a0dd-a4e9f4e58941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:20.058934 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:25:20.105802 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1410/2805] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:25:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:25:20.115840 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b049fb8b-6ad5-44a3-8832-1ea950fe6bc9 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:20.117219 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-b049fb8b-6ad5-44a3-8832-1ea950fe6bc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:20.173417 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b049fb8b-6ad5-44a3-8832-1ea950fe6bc9 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:20.177066 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c88c32-7aad-4286-915d-7b79b2edb625 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:20.177662 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c88c32-7aad-4286-915d-7b79b2edb625 service neutron] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:20.177943 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c88c32-7aad-4286-915d-7b79b2edb625 service neutron] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:20.178219 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c88c32-7aad-4286-915d-7b79b2edb625 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:20.187709 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c88c32-7aad-4286-915d-7b79b2edb625 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:20.189134 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c88c32-7aad-4286-915d-7b79b2edb625 service neutron] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:20.198402 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1396/2806] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:25:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:25:20.463793 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023c4f7f-cc2c-4e10-a0dd-a4e9f4e58941 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:20.465248 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-023c4f7f-cc2c-4e10-a0dd-a4e9f4e58941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:20.477411 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023c4f7f-cc2c-4e10-a0dd-a4e9f4e58941 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:20.482423 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4124ae7-ba32-4aae-a481-b12dc732554b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:20.483574 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4124ae7-ba32-4aae-a481-b12dc732554b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:20.483856 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4124ae7-ba32-4aae-a481-b12dc732554b admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:20.484151 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4124ae7-ba32-4aae-a481-b12dc732554b admin admin] PATH_INFO: `/v3/projects/20dfbfbb3c6c4bada2aa77d3c8a76630` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:20.485710 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4124ae7-ba32-4aae-a481-b12dc732554b admin admin] RBAC: Authorizing `identity:delete_project(project_id=20dfbfbb3c6c4bada2aa77d3c8a76630)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:20.487461 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4124ae7-ba32-4aae-a481-b12dc732554b admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:20.534881 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-b4124ae7-ba32-4aae-a481-b12dc732554b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20dfbfbb3c6c4bada2aa77d3c8a76630', 'request_id': 'req-b4124ae7-ba32-4aae-a481-b12dc732554b'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:20.541070 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-b4124ae7-ba32-4aae-a481-b12dc732554b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20dfbfbb3c6c4bada2aa77d3c8a76630', 'request_id': 'req-b4124ae7-ba32-4aae-a481-b12dc732554b'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:20.549005 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-b4124ae7-ba32-4aae-a481-b12dc732554b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20dfbfbb3c6c4bada2aa77d3c8a76630', 'request_id': 'req-b4124ae7-ba32-4aae-a481-b12dc732554b'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:20.559351 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-b4124ae7-ba32-4aae-a481-b12dc732554b admin admin] The token cache is being invalidate because project 20dfbfbb3c6c4bada2aa77d3c8a76630 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81712) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:25:20.559638 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-b4124ae7-ba32-4aae-a481-b12dc732554b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:20.560260 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-b4124ae7-ba32-4aae-a481-b12dc732554b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:20.562497 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1411/2807] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:25:20 2022] DELETE /identity/v3/projects/20dfbfbb3c6c4bada2aa77d3c8a76630 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:25:20.566419 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c88c32-7aad-4286-915d-7b79b2edb625 service neutron] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:20.575803 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c88c32-7aad-4286-915d-7b79b2edb625 service neutron] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:20.630204 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d59f02-ff6d-460a-8a7b-d8a1716f13e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:20.630817 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d59f02-ff6d-460a-8a7b-d8a1716f13e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:20.631093 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d59f02-ff6d-460a-8a7b-d8a1716f13e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:20.631394 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d59f02-ff6d-460a-8a7b-d8a1716f13e4 admin admin] PATH_INFO: `/v3/users/1a2496c48ef446e2823cea9ee043c283` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:20.632483 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d59f02-ff6d-460a-8a7b-d8a1716f13e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a2496c48ef446e2823cea9ee043c283)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:20.634396 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d59f02-ff6d-460a-8a7b-d8a1716f13e4 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:20.639529 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:25:20.681314 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-99d59f02-ff6d-460a-8a7b-d8a1716f13e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a2496c48ef446e2823cea9ee043c283'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:20.688479 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-99d59f02-ff6d-460a-8a7b-d8a1716f13e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a2496c48ef446e2823cea9ee043c283'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:20.695418 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-99d59f02-ff6d-460a-8a7b-d8a1716f13e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a2496c48ef446e2823cea9ee043c283'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:20.696050 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-99d59f02-ff6d-460a-8a7b-d8a1716f13e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a2496c48ef446e2823cea9ee043c283'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:20.708139 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-99d59f02-ff6d-460a-8a7b-d8a1716f13e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a2496c48ef446e2823cea9ee043c283'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:20.713864 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1397/2808] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:25:20 2022] DELETE /identity/v3/users/1a2496c48ef446e2823cea9ee043c283 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:25:20.989728 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4124ae7-ba32-4aae-a481-b12dc732554b admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:20.991223 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-b4124ae7-ba32-4aae-a481-b12dc732554b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:21.004440 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4124ae7-ba32-4aae-a481-b12dc732554b admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:21.060825 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8f728d-a5f1-43f6-98e6-79b876314f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:21.061690 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8f728d-a5f1-43f6-98e6-79b876314f5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:21.061812 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8f728d-a5f1-43f6-98e6-79b876314f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:21.061984 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8f728d-a5f1-43f6-98e6-79b876314f5b admin admin] PATH_INFO: `/v3/projects/ec0e812ebda84f8b83ab51922cefdbd7` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:21.063084 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8f728d-a5f1-43f6-98e6-79b876314f5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec0e812ebda84f8b83ab51922cefdbd7)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:21.064993 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8f728d-a5f1-43f6-98e6-79b876314f5b admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:21.107979 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:25:21.111523 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-5b8f728d-a5f1-43f6-98e6-79b876314f5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec0e812ebda84f8b83ab51922cefdbd7', 'request_id': 'req-5b8f728d-a5f1-43f6-98e6-79b876314f5b'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:21.118614 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-5b8f728d-a5f1-43f6-98e6-79b876314f5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec0e812ebda84f8b83ab51922cefdbd7', 'request_id': 'req-5b8f728d-a5f1-43f6-98e6-79b876314f5b'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:21.125614 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-5b8f728d-a5f1-43f6-98e6-79b876314f5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec0e812ebda84f8b83ab51922cefdbd7', 'request_id': 'req-5b8f728d-a5f1-43f6-98e6-79b876314f5b'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:21.135578 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-5b8f728d-a5f1-43f6-98e6-79b876314f5b admin admin] The token cache is being invalidate because project ec0e812ebda84f8b83ab51922cefdbd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81712) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:25:21.135863 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-5b8f728d-a5f1-43f6-98e6-79b876314f5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:21.136523 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-5b8f728d-a5f1-43f6-98e6-79b876314f5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:21.138092 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1412/2809] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:25:20 2022] DELETE /identity/v3/projects/ec0e812ebda84f8b83ab51922cefdbd7 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:25:21.143717 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a4b72-582f-4c07-bd7b-20ee6e4d1e65 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:21.144174 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a4b72-582f-4c07-bd7b-20ee6e4d1e65 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:21.144286 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a4b72-582f-4c07-bd7b-20ee6e4d1e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:21.163688 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-327a4b72-582f-4c07-bd7b-20ee6e4d1e65 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:25:21.164882 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-327a4b72-582f-4c07-bd7b-20ee6e4d1e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:21.223427 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1398/2810] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:25:21.231140 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdcd932-ef13-45ec-848b-c7dd129ecd95 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:21.231449 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdcd932-ef13-45ec-848b-c7dd129ecd95 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:21.231744 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdcd932-ef13-45ec-848b-c7dd129ecd95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:21.251828 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-8cdcd932-ef13-45ec-848b-c7dd129ecd95 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:25:21.253070 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-8cdcd932-ef13-45ec-848b-c7dd129ecd95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:21.260856 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1413/2811] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:25:21.265959 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327a4b72-582f-4c07-bd7b-20ee6e4d1e65 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:21.275690 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327a4b72-582f-4c07-bd7b-20ee6e4d1e65 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:21.281426 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df610cd-f115-40fa-9db2-4a5ef34b9155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:21.282025 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df610cd-f115-40fa-9db2-4a5ef34b9155 admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:21.282299 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df610cd-f115-40fa-9db2-4a5ef34b9155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:21.282567 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df610cd-f115-40fa-9db2-4a5ef34b9155 admin admin] PATH_INFO: `/v3/domains` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:21.283268 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df610cd-f115-40fa-9db2-4a5ef34b9155 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81713) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:25:21.283568 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df610cd-f115-40fa-9db2-4a5ef34b9155 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:21.285178 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df610cd-f115-40fa-9db2-4a5ef34b9155 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:21.297101 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1399/2812] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:25:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:25:21.301467 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdcd932-ef13-45ec-848b-c7dd129ecd95 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:21.310635 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdcd932-ef13-45ec-848b-c7dd129ecd95 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:21.315914 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97d3848-b935-4209-a6be-8d3100f5240f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:21.316451 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d3848-b935-4209-a6be-8d3100f5240f admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:21.316730 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d3848-b935-4209-a6be-8d3100f5240f admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:21.317000 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d3848-b935-4209-a6be-8d3100f5240f admin admin] PATH_INFO: `/v3/projects` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:21.317633 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97d3848-b935-4209-a6be-8d3100f5240f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:21.319558 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97d3848-b935-4209-a6be-8d3100f5240f admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:21.346986 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1414/2813] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:25:21 2022] POST /identity/v3/projects => generated 387 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:25:21.351149 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df610cd-f115-40fa-9db2-4a5ef34b9155 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:21.361761 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df610cd-f115-40fa-9db2-4a5ef34b9155 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:21.416623 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4605def-76f1-4b8e-bd6a-50b7f2399f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:21.417217 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4605def-76f1-4b8e-bd6a-50b7f2399f55 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:21.417494 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4605def-76f1-4b8e-bd6a-50b7f2399f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:21.417762 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4605def-76f1-4b8e-bd6a-50b7f2399f55 admin admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:21.418375 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4605def-76f1-4b8e-bd6a-50b7f2399f55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:21.420303 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4605def-76f1-4b8e-bd6a-50b7f2399f55 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:21.440753 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1400/2814] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:25:21 2022] POST /identity/v3/users => generated 294 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:25:21.444975 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97d3848-b935-4209-a6be-8d3100f5240f admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:21.455119 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97d3848-b935-4209-a6be-8d3100f5240f admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:21.459711 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f12d1-133b-496b-aef8-c9ec820eec02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:21.460294 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f12d1-133b-496b-aef8-c9ec820eec02 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:21.460572 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f12d1-133b-496b-aef8-c9ec820eec02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:21.460841 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f12d1-133b-496b-aef8-c9ec820eec02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:21.461470 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3f12d1-133b-496b-aef8-c9ec820eec02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:21.463078 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3f12d1-133b-496b-aef8-c9ec820eec02 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:21.473397 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1415/2815] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:25:21.477184 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4605def-76f1-4b8e-bd6a-50b7f2399f55 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:21.482353 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:25:21.489167 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4605def-76f1-4b8e-bd6a-50b7f2399f55 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:21.494524 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee22241f-5003-405d-976b-fe898d251ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:21.495116 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee22241f-5003-405d-976b-fe898d251ecb admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:21.495429 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee22241f-5003-405d-976b-fe898d251ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:21.495701 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee22241f-5003-405d-976b-fe898d251ecb admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:21.496335 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee22241f-5003-405d-976b-fe898d251ecb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:21.497901 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee22241f-5003-405d-976b-fe898d251ecb admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:21.508263 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1401/2816] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:25:21.512439 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f12d1-133b-496b-aef8-c9ec820eec02 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:21.522791 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f12d1-133b-496b-aef8-c9ec820eec02 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:21.527353 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7848e4e6-2572-4e5a-b2c3-6118cb3f20fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:21.527982 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848e4e6-2572-4e5a-b2c3-6118cb3f20fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:21.528257 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848e4e6-2572-4e5a-b2c3-6118cb3f20fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:21.528525 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848e4e6-2572-4e5a-b2c3-6118cb3f20fc admin admin] PATH_INFO: `/v3/projects/86c40c44c3494741bfd4d22b47d3172f/users/12f3a2f828184698b5ba087e81ad935e/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:21.543438 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7848e4e6-2572-4e5a-b2c3-6118cb3f20fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=86c40c44c3494741bfd4d22b47d3172f, user_id=12f3a2f828184698b5ba087e81ad935e, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:21.546708 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7848e4e6-2572-4e5a-b2c3-6118cb3f20fc admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:21.559484 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1416/2817] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:25:21 2022] PUT /identity/v3/projects/86c40c44c3494741bfd4d22b47d3172f/users/12f3a2f828184698b5ba087e81ad935e/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:25:21.569651 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90655b02-ca9a-47a9-aa74-84778dcf101c None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:21.569873 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90655b02-ca9a-47a9-aa74-84778dcf101c None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:21.570166 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90655b02-ca9a-47a9-aa74-84778dcf101c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:21.597316 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-90655b02-ca9a-47a9-aa74-84778dcf101c None None] MFA Rules not processed for user `12f3a2f828184698b5ba087e81ad935e`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:25:21.598495 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-90655b02-ca9a-47a9-aa74-84778dcf101c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:21.660267 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1402/2818] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:25:21 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:25:21.664482 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7848e4e6-2572-4e5a-b2c3-6118cb3f20fc admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:21.726204 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7848e4e6-2572-4e5a-b2c3-6118cb3f20fc admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:21.730747 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b27fc70-4b30-4e37-a730-79b0361c1c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:21.731364 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b27fc70-4b30-4e37-a730-79b0361c1c7f admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:21.731642 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b27fc70-4b30-4e37-a730-79b0361c1c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:21.731910 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b27fc70-4b30-4e37-a730-79b0361c1c7f admin admin] PATH_INFO: `/v3/projects` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:21.732527 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b27fc70-4b30-4e37-a730-79b0361c1c7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:21.735086 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b27fc70-4b30-4e37-a730-79b0361c1c7f admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:21.761628 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1417/2819] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:25:21 2022] POST /identity/v3/projects => generated 387 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:25:21.766101 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90655b02-ca9a-47a9-aa74-84778dcf101c None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:21.776613 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90655b02-ca9a-47a9-aa74-84778dcf101c None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:21.832316 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc8ce4-e575-47b7-ab1c-15e97abfd3ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:21.832833 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc8ce4-e575-47b7-ab1c-15e97abfd3ec admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:21.833112 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc8ce4-e575-47b7-ab1c-15e97abfd3ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:21.833378 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc8ce4-e575-47b7-ab1c-15e97abfd3ec admin admin] PATH_INFO: `/v3/users` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:21.833989 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffc8ce4-e575-47b7-ab1c-15e97abfd3ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:21.835948 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffc8ce4-e575-47b7-ab1c-15e97abfd3ec admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:21.860641 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1403/2820] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:25:21 2022] POST /identity/v3/users => generated 300 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:25:21.862476 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b27fc70-4b30-4e37-a730-79b0361c1c7f admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:21.872556 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b27fc70-4b30-4e37-a730-79b0361c1c7f admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:21.877303 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0b4c05-2d65-4078-8d77-a466d1375b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:21.877890 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b4c05-2d65-4078-8d77-a466d1375b47 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:21.878162 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b4c05-2d65-4078-8d77-a466d1375b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:21.878431 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b4c05-2d65-4078-8d77-a466d1375b47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:21.879113 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0b4c05-2d65-4078-8d77-a466d1375b47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:21.880931 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0b4c05-2d65-4078-8d77-a466d1375b47 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:21.892024 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1418/2821] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:25:21.896300 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc8ce4-e575-47b7-ab1c-15e97abfd3ec admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:21.907046 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc8ce4-e575-47b7-ab1c-15e97abfd3ec admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:21.911738 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca049797-042c-4b4e-970f-624a1bc5ab0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:21.912376 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca049797-042c-4b4e-970f-624a1bc5ab0d admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:21.912652 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca049797-042c-4b4e-970f-624a1bc5ab0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:21.912923 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca049797-042c-4b4e-970f-624a1bc5ab0d admin admin] PATH_INFO: `/v3/projects/7b3e9604ffe64c07946a8196eaba79ff/users/05274244129f4614a52d3c875e6f1429/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:21.927263 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca049797-042c-4b4e-970f-624a1bc5ab0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b3e9604ffe64c07946a8196eaba79ff, user_id=05274244129f4614a52d3c875e6f1429, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:21.929932 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca049797-042c-4b4e-970f-624a1bc5ab0d admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:21.942841 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1404/2822] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:25:21 2022] PUT /identity/v3/projects/7b3e9604ffe64c07946a8196eaba79ff/users/05274244129f4614a52d3c875e6f1429/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:25:21.946820 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0b4c05-2d65-4078-8d77-a466d1375b47 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:21.957434 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0b4c05-2d65-4078-8d77-a466d1375b47 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:21.999474 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:25:22.016506 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36827e40-6c1c-47a6-a826-d4998f31a839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:22.017109 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36827e40-6c1c-47a6-a826-d4998f31a839 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:22.017389 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36827e40-6c1c-47a6-a826-d4998f31a839 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:22.017661 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36827e40-6c1c-47a6-a826-d4998f31a839 admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:22.018294 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36827e40-6c1c-47a6-a826-d4998f31a839 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:22.019915 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36827e40-6c1c-47a6-a826-d4998f31a839 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:22.029684 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1419/2823] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:25:22.034354 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca049797-042c-4b4e-970f-624a1bc5ab0d admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:22.044564 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca049797-042c-4b4e-970f-624a1bc5ab0d admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:22.049176 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9081f2d0-2e4f-453e-8f80-b3801e0d6e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:22.049846 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9081f2d0-2e4f-453e-8f80-b3801e0d6e5e admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:22.050124 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9081f2d0-2e4f-453e-8f80-b3801e0d6e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:22.050398 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9081f2d0-2e4f-453e-8f80-b3801e0d6e5e admin admin] PATH_INFO: `/v3/projects/7b3e9604ffe64c07946a8196eaba79ff/users/05274244129f4614a52d3c875e6f1429/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:22.052336 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9081f2d0-2e4f-453e-8f80-b3801e0d6e5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b3e9604ffe64c07946a8196eaba79ff, user_id=05274244129f4614a52d3c875e6f1429, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:22.054944 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9081f2d0-2e4f-453e-8f80-b3801e0d6e5e admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:22.064951 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1405/2824] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:25:22 2022] PUT /identity/v3/projects/7b3e9604ffe64c07946a8196eaba79ff/users/05274244129f4614a52d3c875e6f1429/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:25:22.074945 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8efb673-743a-4b12-b203-9f8d7f2f408a None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:22.075454 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8efb673-743a-4b12-b203-9f8d7f2f408a None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:22.075596 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8efb673-743a-4b12-b203-9f8d7f2f408a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:22.103608 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-c8efb673-743a-4b12-b203-9f8d7f2f408a None None] MFA Rules not processed for user `05274244129f4614a52d3c875e6f1429`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:25:22.104735 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-c8efb673-743a-4b12-b203-9f8d7f2f408a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:22.172438 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1420/2825] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:25:22 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:25:22.182876 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9081f2d0-2e4f-453e-8f80-b3801e0d6e5e admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:22.184627 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-9081f2d0-2e4f-453e-8f80-b3801e0d6e5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:22.220399 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:25:22.244747 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9081f2d0-2e4f-453e-8f80-b3801e0d6e5e admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:22.248417 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8ecb19-5b5c-47a9-bd6b-f75e38081ca4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:22.249675 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ecb19-5b5c-47a9-bd6b-f75e38081ca4 service cinder] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:22.249970 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ecb19-5b5c-47a9-bd6b-f75e38081ca4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:22.250235 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ecb19-5b5c-47a9-bd6b-f75e38081ca4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:22.260115 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8ecb19-5b5c-47a9-bd6b-f75e38081ca4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:22.261703 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8ecb19-5b5c-47a9-bd6b-f75e38081ca4 service cinder] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:22.272582 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1406/2826] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:25:22 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:25:22.806159 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8efb673-743a-4b12-b203-9f8d7f2f408a None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:22.866848 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8efb673-743a-4b12-b203-9f8d7f2f408a None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:22.871240 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc2caef-6209-47c3-9a90-df0884a5cc3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:22.871859 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc2caef-6209-47c3-9a90-df0884a5cc3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:22.872138 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc2caef-6209-47c3-9a90-df0884a5cc3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:22.872407 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc2caef-6209-47c3-9a90-df0884a5cc3f admin admin] PATH_INFO: `/v3/users/12f3a2f828184698b5ba087e81ad935e` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:22.873558 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc2caef-6209-47c3-9a90-df0884a5cc3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=12f3a2f828184698b5ba087e81ad935e)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:22.876096 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc2caef-6209-47c3-9a90-df0884a5cc3f admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:22.918914 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-ddc2caef-6209-47c3-9a90-df0884a5cc3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12f3a2f828184698b5ba087e81ad935e'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:22.926589 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-ddc2caef-6209-47c3-9a90-df0884a5cc3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12f3a2f828184698b5ba087e81ad935e'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:22.927187 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-ddc2caef-6209-47c3-9a90-df0884a5cc3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12f3a2f828184698b5ba087e81ad935e'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:22.933910 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-ddc2caef-6209-47c3-9a90-df0884a5cc3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12f3a2f828184698b5ba087e81ad935e'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:22.934525 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-ddc2caef-6209-47c3-9a90-df0884a5cc3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12f3a2f828184698b5ba087e81ad935e'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:22.939022 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:25:22.954077 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1421/2827] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:25:22 2022] DELETE /identity/v3/users/12f3a2f828184698b5ba087e81ad935e => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:25:22.958657 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be1d737-46de-4155-856e-54ec93e21820 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:22.958943 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be1d737-46de-4155-856e-54ec93e21820 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:22.959248 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be1d737-46de-4155-856e-54ec93e21820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:22.977928 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-2be1d737-46de-4155-856e-54ec93e21820 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:25:22.979457 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-2be1d737-46de-4155-856e-54ec93e21820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:23.037547 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1407/2828] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:25:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:25:23.047986 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc2caef-6209-47c3-9a90-df0884a5cc3f admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:23.049355 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-ddc2caef-6209-47c3-9a90-df0884a5cc3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:23.105036 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc2caef-6209-47c3-9a90-df0884a5cc3f admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:23.109445 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d000c2f5-9b47-48bd-a98f-cfe8ee173df0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:23.110041 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d000c2f5-9b47-48bd-a98f-cfe8ee173df0 service neutron] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:23.110320 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d000c2f5-9b47-48bd-a98f-cfe8ee173df0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:23.110856 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d000c2f5-9b47-48bd-a98f-cfe8ee173df0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:23.119886 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d000c2f5-9b47-48bd-a98f-cfe8ee173df0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:23.121305 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d000c2f5-9b47-48bd-a98f-cfe8ee173df0 service neutron] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:23.130905 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1422/2829] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:25:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:25:23.392568 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be1d737-46de-4155-856e-54ec93e21820 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:23.394625 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-2be1d737-46de-4155-856e-54ec93e21820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:23.407184 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be1d737-46de-4155-856e-54ec93e21820 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:23.411784 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd28ce8-40ae-4c03-b722-77ba5ab23050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:23.412413 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd28ce8-40ae-4c03-b722-77ba5ab23050 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:23.412692 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd28ce8-40ae-4c03-b722-77ba5ab23050 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:23.412966 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd28ce8-40ae-4c03-b722-77ba5ab23050 admin admin] PATH_INFO: `/v3/projects/86c40c44c3494741bfd4d22b47d3172f` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:23.414076 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd28ce8-40ae-4c03-b722-77ba5ab23050 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86c40c44c3494741bfd4d22b47d3172f)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:23.416010 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd28ce8-40ae-4c03-b722-77ba5ab23050 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:23.462727 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-efd28ce8-40ae-4c03-b722-77ba5ab23050 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86c40c44c3494741bfd4d22b47d3172f', 'request_id': 'req-efd28ce8-40ae-4c03-b722-77ba5ab23050'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:23.469584 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-efd28ce8-40ae-4c03-b722-77ba5ab23050 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86c40c44c3494741bfd4d22b47d3172f', 'request_id': 'req-efd28ce8-40ae-4c03-b722-77ba5ab23050'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:23.477043 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-efd28ce8-40ae-4c03-b722-77ba5ab23050 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86c40c44c3494741bfd4d22b47d3172f', 'request_id': 'req-efd28ce8-40ae-4c03-b722-77ba5ab23050'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:23.487685 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-efd28ce8-40ae-4c03-b722-77ba5ab23050 admin admin] The token cache is being invalidate because project 86c40c44c3494741bfd4d22b47d3172f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:25:23.488150 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-efd28ce8-40ae-4c03-b722-77ba5ab23050 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:23.488799 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-efd28ce8-40ae-4c03-b722-77ba5ab23050 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:23.490297 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1408/2830] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:25:23 2022] DELETE /identity/v3/projects/86c40c44c3494741bfd4d22b47d3172f => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 12:25:23.494290 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d000c2f5-9b47-48bd-a98f-cfe8ee173df0 service neutron] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:23.504095 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d000c2f5-9b47-48bd-a98f-cfe8ee173df0 service neutron] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:23.561842 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc79ca0-d6ba-40da-aa72-3ad0bfbba564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:23.562466 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc79ca0-d6ba-40da-aa72-3ad0bfbba564 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:23.562743 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc79ca0-d6ba-40da-aa72-3ad0bfbba564 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:23.563011 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc79ca0-d6ba-40da-aa72-3ad0bfbba564 admin admin] PATH_INFO: `/v3/users/05274244129f4614a52d3c875e6f1429` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:23.564146 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc79ca0-d6ba-40da-aa72-3ad0bfbba564 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05274244129f4614a52d3c875e6f1429)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:23.566042 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc79ca0-d6ba-40da-aa72-3ad0bfbba564 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:23.609859 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-8cc79ca0-d6ba-40da-aa72-3ad0bfbba564 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05274244129f4614a52d3c875e6f1429'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:23.617518 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-8cc79ca0-d6ba-40da-aa72-3ad0bfbba564 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05274244129f4614a52d3c875e6f1429'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:23.618131 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-8cc79ca0-d6ba-40da-aa72-3ad0bfbba564 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05274244129f4614a52d3c875e6f1429'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:23.625526 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-8cc79ca0-d6ba-40da-aa72-3ad0bfbba564 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05274244129f4614a52d3c875e6f1429'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:23.626148 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-8cc79ca0-d6ba-40da-aa72-3ad0bfbba564 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05274244129f4614a52d3c875e6f1429'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:23.643156 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1423/2831] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:25:23 2022] DELETE /identity/v3/users/05274244129f4614a52d3c875e6f1429 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:25:23.912958 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd28ce8-40ae-4c03-b722-77ba5ab23050 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:23.914566 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-efd28ce8-40ae-4c03-b722-77ba5ab23050 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:23.928690 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd28ce8-40ae-4c03-b722-77ba5ab23050 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:23.987784 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379e776f-1f02-4c81-be22-dd5eff7f7581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:23.988362 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e776f-1f02-4c81-be22-dd5eff7f7581 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:23.988636 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e776f-1f02-4c81-be22-dd5eff7f7581 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:23.988904 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e776f-1f02-4c81-be22-dd5eff7f7581 admin admin] PATH_INFO: `/v3/projects/7b3e9604ffe64c07946a8196eaba79ff` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:23.990000 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379e776f-1f02-4c81-be22-dd5eff7f7581 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b3e9604ffe64c07946a8196eaba79ff)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:23.991925 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379e776f-1f02-4c81-be22-dd5eff7f7581 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:23.993317 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:25:24.041768 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-379e776f-1f02-4c81-be22-dd5eff7f7581 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b3e9604ffe64c07946a8196eaba79ff', 'request_id': 'req-379e776f-1f02-4c81-be22-dd5eff7f7581'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:24.048588 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-379e776f-1f02-4c81-be22-dd5eff7f7581 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b3e9604ffe64c07946a8196eaba79ff', 'request_id': 'req-379e776f-1f02-4c81-be22-dd5eff7f7581'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:24.055806 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-379e776f-1f02-4c81-be22-dd5eff7f7581 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b3e9604ffe64c07946a8196eaba79ff', 'request_id': 'req-379e776f-1f02-4c81-be22-dd5eff7f7581'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:24.066304 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-379e776f-1f02-4c81-be22-dd5eff7f7581 admin admin] The token cache is being invalidate because project 7b3e9604ffe64c07946a8196eaba79ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:25:24.066593 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-379e776f-1f02-4c81-be22-dd5eff7f7581 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:24.067237 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-379e776f-1f02-4c81-be22-dd5eff7f7581 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:24.068919 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1409/2832] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:25:23 2022] DELETE /identity/v3/projects/7b3e9604ffe64c07946a8196eaba79ff => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:25:24.073551 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93149b1-ab05-4316-b0b7-b153ca520212 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:24.073834 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93149b1-ab05-4316-b0b7-b153ca520212 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:24.074105 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93149b1-ab05-4316-b0b7-b153ca520212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:24.089119 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:25:24.097826 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-f93149b1-ab05-4316-b0b7-b153ca520212 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:25:24.099722 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-f93149b1-ab05-4316-b0b7-b153ca520212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:24.163387 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1424/2833] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:25:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:25:24.171875 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48969cd0-5f74-4315-95b5-0001e9ee6e52 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:24.172155 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48969cd0-5f74-4315-95b5-0001e9ee6e52 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:24.172433 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48969cd0-5f74-4315-95b5-0001e9ee6e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:24.192096 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-48969cd0-5f74-4315-95b5-0001e9ee6e52 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:25:24.193305 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-48969cd0-5f74-4315-95b5-0001e9ee6e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:24.202312 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1410/2834] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:25:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:25:24.206711 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93149b1-ab05-4316-b0b7-b153ca520212 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:24.216470 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93149b1-ab05-4316-b0b7-b153ca520212 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:24.222958 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83b817d-5a6f-438a-b9d6-d50948c3e735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:24.224033 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b817d-5a6f-438a-b9d6-d50948c3e735 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:24.224328 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b817d-5a6f-438a-b9d6-d50948c3e735 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:24.224649 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b817d-5a6f-438a-b9d6-d50948c3e735 admin admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:24.225301 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83b817d-5a6f-438a-b9d6-d50948c3e735 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:25:24.225607 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83b817d-5a6f-438a-b9d6-d50948c3e735 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:24.227226 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83b817d-5a6f-438a-b9d6-d50948c3e735 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:24.239536 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1425/2835] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:25:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:25:24.243677 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48969cd0-5f74-4315-95b5-0001e9ee6e52 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:24.254285 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48969cd0-5f74-4315-95b5-0001e9ee6e52 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:24.259028 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa08d71-747a-4938-8100-c68a88b6540c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:24.259658 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa08d71-747a-4938-8100-c68a88b6540c admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:24.259940 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa08d71-747a-4938-8100-c68a88b6540c admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:24.260212 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa08d71-747a-4938-8100-c68a88b6540c admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:24.260830 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa08d71-747a-4938-8100-c68a88b6540c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:24.262710 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa08d71-747a-4938-8100-c68a88b6540c admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:24.293794 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1411/2836] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:25:24 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:25:24.298044 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83b817d-5a6f-438a-b9d6-d50948c3e735 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:24.308284 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83b817d-5a6f-438a-b9d6-d50948c3e735 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:24.366512 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d925cab-f921-419f-8dc4-9e11ba06e448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:24.367296 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d925cab-f921-419f-8dc4-9e11ba06e448 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:24.367578 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d925cab-f921-419f-8dc4-9e11ba06e448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:24.367921 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d925cab-f921-419f-8dc4-9e11ba06e448 admin admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:24.368555 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d925cab-f921-419f-8dc4-9e11ba06e448 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:24.370455 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d925cab-f921-419f-8dc4-9e11ba06e448 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:24.395321 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1426/2837] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:25:24 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 12:25:24.396448 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa08d71-747a-4938-8100-c68a88b6540c admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:24.406990 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa08d71-747a-4938-8100-c68a88b6540c admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:24.411651 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2514cf0e-df98-48d9-b82b-51509926eaaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:24.412248 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2514cf0e-df98-48d9-b82b-51509926eaaa admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:24.412521 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2514cf0e-df98-48d9-b82b-51509926eaaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:24.412788 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2514cf0e-df98-48d9-b82b-51509926eaaa admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:24.413419 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2514cf0e-df98-48d9-b82b-51509926eaaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:24.414986 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2514cf0e-df98-48d9-b82b-51509926eaaa admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:24.425577 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1412/2838] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 12:25:24.429539 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d925cab-f921-419f-8dc4-9e11ba06e448 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:24.440373 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d925cab-f921-419f-8dc4-9e11ba06e448 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:24.444990 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc9a4ff-82a9-4464-8caf-3f1aef5e46d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:24.445587 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc9a4ff-82a9-4464-8caf-3f1aef5e46d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:24.445862 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc9a4ff-82a9-4464-8caf-3f1aef5e46d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:24.446129 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc9a4ff-82a9-4464-8caf-3f1aef5e46d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:24.446761 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc9a4ff-82a9-4464-8caf-3f1aef5e46d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:24.448384 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc9a4ff-82a9-4464-8caf-3f1aef5e46d6 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:24.460131 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1427/2839] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 12:25:24.464252 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2514cf0e-df98-48d9-b82b-51509926eaaa admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:24.474605 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2514cf0e-df98-48d9-b82b-51509926eaaa admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:24.479315 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abc29e1-8d94-429c-a577-71bc50878a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:24.479961 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc29e1-8d94-429c-a577-71bc50878a9d admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:24.480240 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc29e1-8d94-429c-a577-71bc50878a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:24.480513 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc29e1-8d94-429c-a577-71bc50878a9d admin admin] PATH_INFO: `/v3/projects/ef119e5ce59844a4b942a1b20f2a6214/users/e5c286a6864c4543ba6e68beb5b8295b/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:24.495795 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abc29e1-8d94-429c-a577-71bc50878a9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef119e5ce59844a4b942a1b20f2a6214, user_id=e5c286a6864c4543ba6e68beb5b8295b, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:24.499525 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abc29e1-8d94-429c-a577-71bc50878a9d admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:24.512812 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1413/2840] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:25:24 2022] PUT /identity/v3/projects/ef119e5ce59844a4b942a1b20f2a6214/users/e5c286a6864c4543ba6e68beb5b8295b/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 12:25:24.523869 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8415b146-f580-4c23-b585-07bfbcf7fb94 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:24.524365 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8415b146-f580-4c23-b585-07bfbcf7fb94 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:24.524712 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8415b146-f580-4c23-b585-07bfbcf7fb94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:24.553273 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-8415b146-f580-4c23-b585-07bfbcf7fb94 None None] MFA Rules not processed for user `e5c286a6864c4543ba6e68beb5b8295b`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:25:24.554498 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-8415b146-f580-4c23-b585-07bfbcf7fb94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:24.621408 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1428/2841] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:25:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:25:24.626801 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abc29e1-8d94-429c-a577-71bc50878a9d admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:24.688845 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abc29e1-8d94-429c-a577-71bc50878a9d admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:24.693710 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34338fef-63e4-4be9-a5a8-6db6cbb15d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:24.694300 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34338fef-63e4-4be9-a5a8-6db6cbb15d93 admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:24.694576 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34338fef-63e4-4be9-a5a8-6db6cbb15d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:24.694844 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34338fef-63e4-4be9-a5a8-6db6cbb15d93 admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:24.695492 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34338fef-63e4-4be9-a5a8-6db6cbb15d93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:24.697379 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34338fef-63e4-4be9-a5a8-6db6cbb15d93 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:24.723264 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1414/2842] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:25:24 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:25:24.727360 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8415b146-f580-4c23-b585-07bfbcf7fb94 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:24.732714 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:25:24.743887 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8415b146-f580-4c23-b585-07bfbcf7fb94 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:24.799577 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5229212b-fab6-4d44-9e37-0f354891b0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:24.800177 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5229212b-fab6-4d44-9e37-0f354891b0b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:24.800451 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5229212b-fab6-4d44-9e37-0f354891b0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:24.800718 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5229212b-fab6-4d44-9e37-0f354891b0b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:24.801326 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5229212b-fab6-4d44-9e37-0f354891b0b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:24.803238 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5229212b-fab6-4d44-9e37-0f354891b0b7 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:24.823972 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1429/2843] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:25:24 2022] POST /identity/v3/users => generated 299 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:25:24.828188 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34338fef-63e4-4be9-a5a8-6db6cbb15d93 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:24.838256 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34338fef-63e4-4be9-a5a8-6db6cbb15d93 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:24.842834 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f9b8ab-9bf2-466b-a41d-43a325c44a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:24.843473 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f9b8ab-9bf2-466b-a41d-43a325c44a6c admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:24.843747 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f9b8ab-9bf2-466b-a41d-43a325c44a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:24.844015 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f9b8ab-9bf2-466b-a41d-43a325c44a6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:24.844643 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f9b8ab-9bf2-466b-a41d-43a325c44a6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:24.846210 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f9b8ab-9bf2-466b-a41d-43a325c44a6c admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:24.851262 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:25:24.857501 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1415/2844] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 26 12:25:24.862180 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5229212b-fab6-4d44-9e37-0f354891b0b7 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:24.871981 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5229212b-fab6-4d44-9e37-0f354891b0b7 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:24.877104 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c0aa41-2ea4-4020-a753-35c338316474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:24.877735 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c0aa41-2ea4-4020-a753-35c338316474 admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:24.878012 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c0aa41-2ea4-4020-a753-35c338316474 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:24.878282 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c0aa41-2ea4-4020-a753-35c338316474 admin admin] PATH_INFO: `/v3/projects/9c9f4b4e5208402b8b597c4e77d322b7/users/4411743479ad47fe81cf41d1774bc560/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:24.892570 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c0aa41-2ea4-4020-a753-35c338316474 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c9f4b4e5208402b8b597c4e77d322b7, user_id=4411743479ad47fe81cf41d1774bc560, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:24.895184 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c0aa41-2ea4-4020-a753-35c338316474 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:24.907494 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1430/2845] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:25:24 2022] PUT /identity/v3/projects/9c9f4b4e5208402b8b597c4e77d322b7/users/4411743479ad47fe81cf41d1774bc560/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:25:24.911396 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f9b8ab-9bf2-466b-a41d-43a325c44a6c admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:24.921872 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f9b8ab-9bf2-466b-a41d-43a325c44a6c admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:24.976251 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fef5c7-14b2-44e6-bac8-20d5aeb3119d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:24.976847 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fef5c7-14b2-44e6-bac8-20d5aeb3119d admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:24.977128 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fef5c7-14b2-44e6-bac8-20d5aeb3119d admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:24.977400 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fef5c7-14b2-44e6-bac8-20d5aeb3119d admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:24.978054 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fef5c7-14b2-44e6-bac8-20d5aeb3119d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:24.979682 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fef5c7-14b2-44e6-bac8-20d5aeb3119d admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:24.990619 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1416/2846] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:25:24.994848 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c0aa41-2ea4-4020-a753-35c338316474 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:25.005927 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c0aa41-2ea4-4020-a753-35c338316474 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:25.010467 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff7027-c073-44b1-b0c7-a667ec68731b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:25.011108 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff7027-c073-44b1-b0c7-a667ec68731b admin admin] REQUEST_METHOD: `PUT` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:25.011423 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff7027-c073-44b1-b0c7-a667ec68731b admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:25.011721 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff7027-c073-44b1-b0c7-a667ec68731b admin admin] PATH_INFO: `/v3/projects/9c9f4b4e5208402b8b597c4e77d322b7/users/4411743479ad47fe81cf41d1774bc560/roles/70ec1ba484714c8ca3adb8fe5845107c` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:25.013592 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ff7027-c073-44b1-b0c7-a667ec68731b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c9f4b4e5208402b8b597c4e77d322b7, user_id=4411743479ad47fe81cf41d1774bc560, role_id=70ec1ba484714c8ca3adb8fe5845107c)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:25.016296 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ff7027-c073-44b1-b0c7-a667ec68731b admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:25.026356 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1431/2847] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:25:24 2022] PUT /identity/v3/projects/9c9f4b4e5208402b8b597c4e77d322b7/users/4411743479ad47fe81cf41d1774bc560/roles/70ec1ba484714c8ca3adb8fe5845107c => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 26 12:25:25.036698 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef2889-6ecd-413a-a3c0-a091d9a54738 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:25.036985 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef2889-6ecd-413a-a3c0-a091d9a54738 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:25.037260 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef2889-6ecd-413a-a3c0-a091d9a54738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:25.065071 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-8bef2889-6ecd-413a-a3c0-a091d9a54738 None None] MFA Rules not processed for user `4411743479ad47fe81cf41d1774bc560`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:25:25.066261 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-8bef2889-6ecd-413a-a3c0-a091d9a54738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:25.131703 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1417/2848] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:25:25 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:25:25.142171 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff7027-c073-44b1-b0c7-a667ec68731b admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:25.143922 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-71ff7027-c073-44b1-b0c7-a667ec68731b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:25.200032 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff7027-c073-44b1-b0c7-a667ec68731b admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:25.203882 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0eef7c-b76c-4021-8954-cd30ef304bd6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:25.204470 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0eef7c-b76c-4021-8954-cd30ef304bd6 service cinder] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:25.204744 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0eef7c-b76c-4021-8954-cd30ef304bd6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:25.205010 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0eef7c-b76c-4021-8954-cd30ef304bd6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:25.259866 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0eef7c-b76c-4021-8954-cd30ef304bd6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:25.261233 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0eef7c-b76c-4021-8954-cd30ef304bd6 service cinder] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:25.269990 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1432/2849] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:25:25 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:25:29.450333 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bef2889-6ecd-413a-a3c0-a091d9a54738 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:29.451801 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-8bef2889-6ecd-413a-a3c0-a091d9a54738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:29.466220 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bef2889-6ecd-413a-a3c0-a091d9a54738 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:29.470565 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d06f4b-8454-482d-ac2e-e5dfac47fb0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:29.471224 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d06f4b-8454-482d-ac2e-e5dfac47fb0e service cinder] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:29.471511 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d06f4b-8454-482d-ac2e-e5dfac47fb0e service cinder] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:29.471805 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d06f4b-8454-482d-ac2e-e5dfac47fb0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:29.481209 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d06f4b-8454-482d-ac2e-e5dfac47fb0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:29.482632 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d06f4b-8454-482d-ac2e-e5dfac47fb0e service cinder] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:29.493012 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1418/2850] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:25:29 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:25:34.368836 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0eef7c-b76c-4021-8954-cd30ef304bd6 service cinder] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:34.378828 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:25:34.430362 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0eef7c-b76c-4021-8954-cd30ef304bd6 service cinder] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:34.434816 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13083691-aa48-46a1-a4d4-c07b6486ab53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:34.435458 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13083691-aa48-46a1-a4d4-c07b6486ab53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:34.435744 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13083691-aa48-46a1-a4d4-c07b6486ab53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:34.436016 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13083691-aa48-46a1-a4d4-c07b6486ab53 admin admin] PATH_INFO: `/v3/users/e5c286a6864c4543ba6e68beb5b8295b` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:34.437658 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13083691-aa48-46a1-a4d4-c07b6486ab53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5c286a6864c4543ba6e68beb5b8295b)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:34.439600 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13083691-aa48-46a1-a4d4-c07b6486ab53 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:34.484402 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-13083691-aa48-46a1-a4d4-c07b6486ab53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5c286a6864c4543ba6e68beb5b8295b'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:34.491886 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-13083691-aa48-46a1-a4d4-c07b6486ab53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5c286a6864c4543ba6e68beb5b8295b'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:34.493031 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-13083691-aa48-46a1-a4d4-c07b6486ab53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5c286a6864c4543ba6e68beb5b8295b'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:34.500742 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-13083691-aa48-46a1-a4d4-c07b6486ab53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5c286a6864c4543ba6e68beb5b8295b'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:34.501364 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-13083691-aa48-46a1-a4d4-c07b6486ab53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5c286a6864c4543ba6e68beb5b8295b'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:34.518298 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1433/2851] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:25:34 2022] DELETE /identity/v3/users/e5c286a6864c4543ba6e68beb5b8295b => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:25:34.522837 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961258b0-967e-4418-b10c-db849d87e812 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:34.523393 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961258b0-967e-4418-b10c-db849d87e812 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:34.523393 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961258b0-967e-4418-b10c-db849d87e812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:34.541703 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-961258b0-967e-4418-b10c-db849d87e812 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:25:34.542877 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-961258b0-967e-4418-b10c-db849d87e812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:34.599233 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1419/2852] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:25:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:25:34.610152 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13083691-aa48-46a1-a4d4-c07b6486ab53 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:34.611861 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-13083691-aa48-46a1-a4d4-c07b6486ab53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:34.671443 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13083691-aa48-46a1-a4d4-c07b6486ab53 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:34.675013 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba5bc7d-4618-4cd4-b27e-b9db022506d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:34.675641 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba5bc7d-4618-4cd4-b27e-b9db022506d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:34.675917 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba5bc7d-4618-4cd4-b27e-b9db022506d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:34.676183 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba5bc7d-4618-4cd4-b27e-b9db022506d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:34.684863 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba5bc7d-4618-4cd4-b27e-b9db022506d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:34.686366 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba5bc7d-4618-4cd4-b27e-b9db022506d3 service neutron] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:34.696162 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1434/2853] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:25:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:25:34.962592 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961258b0-967e-4418-b10c-db849d87e812 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:34.964189 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-961258b0-967e-4418-b10c-db849d87e812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:34.979861 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961258b0-967e-4418-b10c-db849d87e812 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:34.985828 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0a6138-2af9-4790-9f6d-ab1532af5fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:34.986670 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0a6138-2af9-4790-9f6d-ab1532af5fa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:34.987381 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0a6138-2af9-4790-9f6d-ab1532af5fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:34.988135 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0a6138-2af9-4790-9f6d-ab1532af5fa9 admin admin] PATH_INFO: `/v3/projects/ef119e5ce59844a4b942a1b20f2a6214` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:34.989047 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0a6138-2af9-4790-9f6d-ab1532af5fa9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef119e5ce59844a4b942a1b20f2a6214)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:34.991237 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0a6138-2af9-4790-9f6d-ab1532af5fa9 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:34.994830 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=100 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:25:35.042496 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-5f0a6138-2af9-4790-9f6d-ab1532af5fa9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef119e5ce59844a4b942a1b20f2a6214', 'request_id': 'req-5f0a6138-2af9-4790-9f6d-ab1532af5fa9'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:35.049181 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-5f0a6138-2af9-4790-9f6d-ab1532af5fa9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef119e5ce59844a4b942a1b20f2a6214', 'request_id': 'req-5f0a6138-2af9-4790-9f6d-ab1532af5fa9'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:35.056473 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-5f0a6138-2af9-4790-9f6d-ab1532af5fa9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef119e5ce59844a4b942a1b20f2a6214', 'request_id': 'req-5f0a6138-2af9-4790-9f6d-ab1532af5fa9'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:35.066973 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-5f0a6138-2af9-4790-9f6d-ab1532af5fa9 admin admin] The token cache is being invalidate because project ef119e5ce59844a4b942a1b20f2a6214 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:25:35.067297 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-5f0a6138-2af9-4790-9f6d-ab1532af5fa9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:35.067930 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-5f0a6138-2af9-4790-9f6d-ab1532af5fa9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:35.069410 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1420/2854] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:25:34 2022] DELETE /identity/v3/projects/ef119e5ce59844a4b942a1b20f2a6214 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:25:35.073514 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba5bc7d-4618-4cd4-b27e-b9db022506d3 service neutron] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:35.083370 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba5bc7d-4618-4cd4-b27e-b9db022506d3 service neutron] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:35.113765 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:25:35.142868 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf4bf86-9525-41ab-b750-75ff5e97e789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:35.143460 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4bf86-9525-41ab-b750-75ff5e97e789 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:35.143741 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4bf86-9525-41ab-b750-75ff5e97e789 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:35.144013 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4bf86-9525-41ab-b750-75ff5e97e789 admin admin] PATH_INFO: `/v3/users/4411743479ad47fe81cf41d1774bc560` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:35.145157 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf4bf86-9525-41ab-b750-75ff5e97e789 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4411743479ad47fe81cf41d1774bc560)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:35.147066 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf4bf86-9525-41ab-b750-75ff5e97e789 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:35.192023 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-cdf4bf86-9525-41ab-b750-75ff5e97e789 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4411743479ad47fe81cf41d1774bc560'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:35.199090 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-cdf4bf86-9525-41ab-b750-75ff5e97e789 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4411743479ad47fe81cf41d1774bc560'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:35.199773 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-cdf4bf86-9525-41ab-b750-75ff5e97e789 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4411743479ad47fe81cf41d1774bc560'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:35.206694 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-cdf4bf86-9525-41ab-b750-75ff5e97e789 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4411743479ad47fe81cf41d1774bc560'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:35.207353 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-cdf4bf86-9525-41ab-b750-75ff5e97e789 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4411743479ad47fe81cf41d1774bc560'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:35.224560 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1435/2855] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:25:35 2022] DELETE /identity/v3/users/4411743479ad47fe81cf41d1774bc560 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:25:35.498957 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0a6138-2af9-4790-9f6d-ab1532af5fa9 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:35.500905 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-5f0a6138-2af9-4790-9f6d-ab1532af5fa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:35.515447 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0a6138-2af9-4790-9f6d-ab1532af5fa9 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:35.572344 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f64a36-9ba0-4f0d-a570-62d3c3d3c5cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:35.573020 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f64a36-9ba0-4f0d-a570-62d3c3d3c5cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:35.573229 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f64a36-9ba0-4f0d-a570-62d3c3d3c5cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:35.573497 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f64a36-9ba0-4f0d-a570-62d3c3d3c5cb admin admin] PATH_INFO: `/v3/projects/9c9f4b4e5208402b8b597c4e77d322b7` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:35.574604 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f64a36-9ba0-4f0d-a570-62d3c3d3c5cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c9f4b4e5208402b8b597c4e77d322b7)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:35.576644 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f64a36-9ba0-4f0d-a570-62d3c3d3c5cb admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:35.621886 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-e4f64a36-9ba0-4f0d-a570-62d3c3d3c5cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c9f4b4e5208402b8b597c4e77d322b7', 'request_id': 'req-e4f64a36-9ba0-4f0d-a570-62d3c3d3c5cb'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:35.627957 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-e4f64a36-9ba0-4f0d-a570-62d3c3d3c5cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c9f4b4e5208402b8b597c4e77d322b7', 'request_id': 'req-e4f64a36-9ba0-4f0d-a570-62d3c3d3c5cb'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:35.635187 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-e4f64a36-9ba0-4f0d-a570-62d3c3d3c5cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c9f4b4e5208402b8b597c4e77d322b7', 'request_id': 'req-e4f64a36-9ba0-4f0d-a570-62d3c3d3c5cb'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:35.646969 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-e4f64a36-9ba0-4f0d-a570-62d3c3d3c5cb admin admin] The token cache is being invalidate because project 9c9f4b4e5208402b8b597c4e77d322b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:25:35.647365 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-e4f64a36-9ba0-4f0d-a570-62d3c3d3c5cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:35.648048 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-e4f64a36-9ba0-4f0d-a570-62d3c3d3c5cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:35.649835 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1421/2856] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:25:35 2022] DELETE /identity/v3/projects/9c9f4b4e5208402b8b597c4e77d322b7 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:25:35.656714 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87cd99a-ab1f-4b36-b187-4e86fdbfdd72 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:35.657192 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87cd99a-ab1f-4b36-b187-4e86fdbfdd72 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:35.657301 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87cd99a-ab1f-4b36-b187-4e86fdbfdd72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:35.676252 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-e87cd99a-ab1f-4b36-b187-4e86fdbfdd72 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:25:35.677377 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-e87cd99a-ab1f-4b36-b187-4e86fdbfdd72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:35.735867 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1436/2857] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:25:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:25:35.744295 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb0c34-0189-4082-ac1c-23e0b5951829 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:35.744584 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb0c34-0189-4082-ac1c-23e0b5951829 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:35.744865 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb0c34-0189-4082-ac1c-23e0b5951829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:35.766009 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-f5fb0c34-0189-4082-ac1c-23e0b5951829 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:25:35.767247 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-f5fb0c34-0189-4082-ac1c-23e0b5951829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:35.775247 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1422/2858] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:25:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:25:35.779451 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87cd99a-ab1f-4b36-b187-4e86fdbfdd72 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:35.789293 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87cd99a-ab1f-4b36-b187-4e86fdbfdd72 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:35.793980 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d309165-cc75-4b93-a55f-fccab035fae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:35.794600 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d309165-cc75-4b93-a55f-fccab035fae8 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:35.794877 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d309165-cc75-4b93-a55f-fccab035fae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:35.795147 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d309165-cc75-4b93-a55f-fccab035fae8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:35.795846 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d309165-cc75-4b93-a55f-fccab035fae8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:25:35.796142 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d309165-cc75-4b93-a55f-fccab035fae8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:35.797728 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d309165-cc75-4b93-a55f-fccab035fae8 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:35.809078 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1437/2859] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:25:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 26 12:25:35.814132 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fb0c34-0189-4082-ac1c-23e0b5951829 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:35.823309 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fb0c34-0189-4082-ac1c-23e0b5951829 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:35.828869 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4418116e-74b4-4199-a644-2d8aaa9d0bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:35.829485 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4418116e-74b4-4199-a644-2d8aaa9d0bab admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:35.829766 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4418116e-74b4-4199-a644-2d8aaa9d0bab admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:35.830040 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4418116e-74b4-4199-a644-2d8aaa9d0bab admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:35.830654 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4418116e-74b4-4199-a644-2d8aaa9d0bab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:35.832588 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4418116e-74b4-4199-a644-2d8aaa9d0bab admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:35.847392 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:25:35.861485 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1423/2860] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:25:35 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:25:35.865631 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d309165-cc75-4b93-a55f-fccab035fae8 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:35.876208 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d309165-cc75-4b93-a55f-fccab035fae8 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:35.909561 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:25:35.933803 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78bf594-1e30-4979-8f22-06d7200a5c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:35.934320 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78bf594-1e30-4979-8f22-06d7200a5c33 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:35.934597 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78bf594-1e30-4979-8f22-06d7200a5c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:35.934867 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78bf594-1e30-4979-8f22-06d7200a5c33 admin admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:35.935522 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78bf594-1e30-4979-8f22-06d7200a5c33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:35.937912 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78bf594-1e30-4979-8f22-06d7200a5c33 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:35.958311 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1438/2861] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:25:35 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:25:35.962397 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4418116e-74b4-4199-a644-2d8aaa9d0bab admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:35.972767 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4418116e-74b4-4199-a644-2d8aaa9d0bab admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:35.977287 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febb0fac-a508-4fda-a7ab-ea9756b7a5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:35.977875 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febb0fac-a508-4fda-a7ab-ea9756b7a5bc admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:35.978150 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febb0fac-a508-4fda-a7ab-ea9756b7a5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:35.978418 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febb0fac-a508-4fda-a7ab-ea9756b7a5bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:35.979048 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febb0fac-a508-4fda-a7ab-ea9756b7a5bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:35.980713 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febb0fac-a508-4fda-a7ab-ea9756b7a5bc admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:35.990661 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1424/2862] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:25:35.994570 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78bf594-1e30-4979-8f22-06d7200a5c33 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:36.005292 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78bf594-1e30-4979-8f22-06d7200a5c33 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:36.009970 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff9b7b4-c57b-4c58-8d8a-33d8c352375c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:36.010566 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff9b7b4-c57b-4c58-8d8a-33d8c352375c admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:36.010846 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff9b7b4-c57b-4c58-8d8a-33d8c352375c admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:36.011119 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff9b7b4-c57b-4c58-8d8a-33d8c352375c admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:36.011790 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff9b7b4-c57b-4c58-8d8a-33d8c352375c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:36.013369 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff9b7b4-c57b-4c58-8d8a-33d8c352375c admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:36.023772 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1439/2863] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:25:36.027788 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febb0fac-a508-4fda-a7ab-ea9756b7a5bc admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:36.038148 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febb0fac-a508-4fda-a7ab-ea9756b7a5bc admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:36.042814 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ff654a-1e52-4028-944d-0be54b5befa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:36.043496 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ff654a-1e52-4028-944d-0be54b5befa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:36.043780 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ff654a-1e52-4028-944d-0be54b5befa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:36.044056 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ff654a-1e52-4028-944d-0be54b5befa7 admin admin] PATH_INFO: `/v3/projects/e48a16d3239544b1b9026e9ab0bf4348/users/58a0360725bd4d2aac5b180305c7ac08/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:36.058979 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ff654a-1e52-4028-944d-0be54b5befa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e48a16d3239544b1b9026e9ab0bf4348, user_id=58a0360725bd4d2aac5b180305c7ac08, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:36.062691 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ff654a-1e52-4028-944d-0be54b5befa7 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:36.075113 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1425/2864] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:25:36 2022] PUT /identity/v3/projects/e48a16d3239544b1b9026e9ab0bf4348/users/58a0360725bd4d2aac5b180305c7ac08/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:25:36.086338 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00afb9b0-7dcf-406f-8cc7-aa4d59e116e1 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:36.086630 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00afb9b0-7dcf-406f-8cc7-aa4d59e116e1 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:36.086908 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00afb9b0-7dcf-406f-8cc7-aa4d59e116e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:36.115822 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-00afb9b0-7dcf-406f-8cc7-aa4d59e116e1 None None] MFA Rules not processed for user `58a0360725bd4d2aac5b180305c7ac08`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:25:36.117016 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-00afb9b0-7dcf-406f-8cc7-aa4d59e116e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:36.180254 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1440/2865] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:25:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:25:36.192218 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ff654a-1e52-4028-944d-0be54b5befa7 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:36.193901 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-97ff654a-1e52-4028-944d-0be54b5befa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:36.249792 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ff654a-1e52-4028-944d-0be54b5befa7 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:36.253544 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f646d-f585-4b2d-94bb-73989666a46a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:36.254137 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f646d-f585-4b2d-94bb-73989666a46a service cinder] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:36.254413 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f646d-f585-4b2d-94bb-73989666a46a service cinder] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:36.254684 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f646d-f585-4b2d-94bb-73989666a46a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:36.262731 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2f646d-f585-4b2d-94bb-73989666a46a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:36.264169 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2f646d-f585-4b2d-94bb-73989666a46a service cinder] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:36.272707 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1426/2866] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:25:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:25:46.177299 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=49,keystone:INSERT=3 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:25:46.268828 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:INSERT=2,keystone:SELECT=43 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:25:55.861713 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00afb9b0-7dcf-406f-8cc7-aa4d59e116e1 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:55.921354 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00afb9b0-7dcf-406f-8cc7-aa4d59e116e1 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:55.926338 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51ce85f-6533-48ec-8ab3-b5c04b892f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:55.926956 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51ce85f-6533-48ec-8ab3-b5c04b892f74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:55.927258 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51ce85f-6533-48ec-8ab3-b5c04b892f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:55.927530 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51ce85f-6533-48ec-8ab3-b5c04b892f74 admin admin] PATH_INFO: `/v3/users/58a0360725bd4d2aac5b180305c7ac08` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:55.928628 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51ce85f-6533-48ec-8ab3-b5c04b892f74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58a0360725bd4d2aac5b180305c7ac08)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:55.930520 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51ce85f-6533-48ec-8ab3-b5c04b892f74 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:55.975435 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-a51ce85f-6533-48ec-8ab3-b5c04b892f74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58a0360725bd4d2aac5b180305c7ac08'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:55.982556 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-a51ce85f-6533-48ec-8ab3-b5c04b892f74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58a0360725bd4d2aac5b180305c7ac08'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:55.983174 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-a51ce85f-6533-48ec-8ab3-b5c04b892f74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58a0360725bd4d2aac5b180305c7ac08'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:55.991086 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-a51ce85f-6533-48ec-8ab3-b5c04b892f74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58a0360725bd4d2aac5b180305c7ac08'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:55.991680 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-a51ce85f-6533-48ec-8ab3-b5c04b892f74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58a0360725bd4d2aac5b180305c7ac08'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:56.008596 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1441/2867] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:25:55 2022] DELETE /identity/v3/users/58a0360725bd4d2aac5b180305c7ac08 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:25:56.013127 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f624fa82-e40b-4185-b572-32157d177530 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:56.013616 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f624fa82-e40b-4185-b572-32157d177530 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:56.013727 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f624fa82-e40b-4185-b572-32157d177530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:56.032793 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-f624fa82-e40b-4185-b572-32157d177530 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:25:56.033976 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-f624fa82-e40b-4185-b572-32157d177530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:56.090378 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1427/2868] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:25:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:25:56.100467 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51ce85f-6533-48ec-8ab3-b5c04b892f74 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:56.101855 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-a51ce85f-6533-48ec-8ab3-b5c04b892f74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:56.159282 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51ce85f-6533-48ec-8ab3-b5c04b892f74 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:56.162822 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcf0a1a-f5ac-4ecd-8ee3-70fae393b1ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:56.163439 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcf0a1a-f5ac-4ecd-8ee3-70fae393b1ab service neutron] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:56.163714 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcf0a1a-f5ac-4ecd-8ee3-70fae393b1ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:56.163986 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcf0a1a-f5ac-4ecd-8ee3-70fae393b1ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:56.173552 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcf0a1a-f5ac-4ecd-8ee3-70fae393b1ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:56.174962 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcf0a1a-f5ac-4ecd-8ee3-70fae393b1ab service neutron] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:56.184168 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1442/2869] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:25:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:25:56.455129 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f624fa82-e40b-4185-b572-32157d177530 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:56.456678 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-f624fa82-e40b-4185-b572-32157d177530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:56.469518 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f624fa82-e40b-4185-b572-32157d177530 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:56.474231 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db7b9ff-62e4-46f9-a231-04a92e0854b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:56.474862 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db7b9ff-62e4-46f9-a231-04a92e0854b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:56.475155 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db7b9ff-62e4-46f9-a231-04a92e0854b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:56.475450 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db7b9ff-62e4-46f9-a231-04a92e0854b7 admin admin] PATH_INFO: `/v3/projects/e48a16d3239544b1b9026e9ab0bf4348` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:56.476583 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db7b9ff-62e4-46f9-a231-04a92e0854b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e48a16d3239544b1b9026e9ab0bf4348)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:56.478513 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db7b9ff-62e4-46f9-a231-04a92e0854b7 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:56.525823 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-3db7b9ff-62e4-46f9-a231-04a92e0854b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e48a16d3239544b1b9026e9ab0bf4348', 'request_id': 'req-3db7b9ff-62e4-46f9-a231-04a92e0854b7'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:56.532427 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-3db7b9ff-62e4-46f9-a231-04a92e0854b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e48a16d3239544b1b9026e9ab0bf4348', 'request_id': 'req-3db7b9ff-62e4-46f9-a231-04a92e0854b7'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:56.539662 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-3db7b9ff-62e4-46f9-a231-04a92e0854b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e48a16d3239544b1b9026e9ab0bf4348', 'request_id': 'req-3db7b9ff-62e4-46f9-a231-04a92e0854b7'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:56.550156 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-3db7b9ff-62e4-46f9-a231-04a92e0854b7 admin admin] The token cache is being invalidate because project e48a16d3239544b1b9026e9ab0bf4348 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81713) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:25:56.550443 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-3db7b9ff-62e4-46f9-a231-04a92e0854b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:56.551048 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-3db7b9ff-62e4-46f9-a231-04a92e0854b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:25:56.552535 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1428/2870] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:25:56 2022] DELETE /identity/v3/projects/e48a16d3239544b1b9026e9ab0bf4348 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:25:56.557295 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb35983-e957-4afc-b62b-2691b8bb6200 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:56.557582 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb35983-e957-4afc-b62b-2691b8bb6200 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:56.557855 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb35983-e957-4afc-b62b-2691b8bb6200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:56.577317 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-3bb35983-e957-4afc-b62b-2691b8bb6200 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:25:56.578756 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-3bb35983-e957-4afc-b62b-2691b8bb6200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:56.595956 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:25:56.639386 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1443/2871] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:25:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:25:56.648128 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2513d81a-8138-4439-a7a0-717530757b7c None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:56.648416 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2513d81a-8138-4439-a7a0-717530757b7c None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:56.648688 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2513d81a-8138-4439-a7a0-717530757b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:56.668053 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-2513d81a-8138-4439-a7a0-717530757b7c None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:25:56.669229 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-2513d81a-8138-4439-a7a0-717530757b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:56.677767 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1429/2872] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:25:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:25:56.681844 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb35983-e957-4afc-b62b-2691b8bb6200 None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:56.691496 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb35983-e957-4afc-b62b-2691b8bb6200 None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:56.695985 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21943630-80f1-4677-ada8-1aa59374f51b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:56.696569 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21943630-80f1-4677-ada8-1aa59374f51b admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:56.696845 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21943630-80f1-4677-ada8-1aa59374f51b admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:56.697115 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21943630-80f1-4677-ada8-1aa59374f51b admin admin] PATH_INFO: `/v3/domains` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:56.697751 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21943630-80f1-4677-ada8-1aa59374f51b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81712) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 26 12:25:56.698051 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21943630-80f1-4677-ada8-1aa59374f51b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:56.699678 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21943630-80f1-4677-ada8-1aa59374f51b admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:56.711033 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1444/2873] 10.222.0.61 () {58 vars in 1153 bytes} [Fri Aug 26 12:25:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 26 12:25:56.714978 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2513d81a-8138-4439-a7a0-717530757b7c None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:56.724459 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2513d81a-8138-4439-a7a0-717530757b7c None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:56.729030 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25c777a-fc2f-4105-a3ce-be7dffd8949e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:56.729607 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25c777a-fc2f-4105-a3ce-be7dffd8949e admin admin] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:56.729883 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25c777a-fc2f-4105-a3ce-be7dffd8949e admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:56.730149 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25c777a-fc2f-4105-a3ce-be7dffd8949e admin admin] PATH_INFO: `/v3/projects` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:56.730757 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25c777a-fc2f-4105-a3ce-be7dffd8949e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:56.732681 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25c777a-fc2f-4105-a3ce-be7dffd8949e admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:56.760210 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1430/2874] 10.222.0.61 () {60 vars in 1153 bytes} [Fri Aug 26 12:25:56 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 26 12:25:56.764352 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21943630-80f1-4677-ada8-1aa59374f51b admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:56.774382 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21943630-80f1-4677-ada8-1aa59374f51b admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:56.831282 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cadad28-2f6e-4925-b04f-5d8f1e8beb99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:56.831837 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cadad28-2f6e-4925-b04f-5d8f1e8beb99 admin admin] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:56.832117 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cadad28-2f6e-4925-b04f-5d8f1e8beb99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:56.832390 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cadad28-2f6e-4925-b04f-5d8f1e8beb99 admin admin] PATH_INFO: `/v3/users` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:56.833005 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cadad28-2f6e-4925-b04f-5d8f1e8beb99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:56.835023 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cadad28-2f6e-4925-b04f-5d8f1e8beb99 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:56.855907 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1445/2875] 10.222.0.61 () {60 vars in 1144 bytes} [Fri Aug 26 12:25:56 2022] POST /identity/v3/users => generated 286 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 26 12:25:56.861212 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25c777a-fc2f-4105-a3ce-be7dffd8949e admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:56.871426 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25c777a-fc2f-4105-a3ce-be7dffd8949e admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:56.876508 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5558e3b-a19d-4b5a-9a0d-bd6cef8065db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:56.877101 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5558e3b-a19d-4b5a-9a0d-bd6cef8065db admin admin] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:56.877384 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5558e3b-a19d-4b5a-9a0d-bd6cef8065db admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:56.877659 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5558e3b-a19d-4b5a-9a0d-bd6cef8065db admin admin] PATH_INFO: `/v3/roles` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:56.878328 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5558e3b-a19d-4b5a-9a0d-bd6cef8065db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:56.879965 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5558e3b-a19d-4b5a-9a0d-bd6cef8065db admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:56.889674 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1431/2876] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:25:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:25:56.893491 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cadad28-2f6e-4925-b04f-5d8f1e8beb99 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:56.903255 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cadad28-2f6e-4925-b04f-5d8f1e8beb99 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:56.908355 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99ae97b-4ecd-42f9-bf20-d4ee9d38b826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:56.908929 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ae97b-4ecd-42f9-bf20-d4ee9d38b826 admin admin] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:56.909204 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ae97b-4ecd-42f9-bf20-d4ee9d38b826 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:56.909471 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ae97b-4ecd-42f9-bf20-d4ee9d38b826 admin admin] PATH_INFO: `/v3/roles` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:56.910099 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99ae97b-4ecd-42f9-bf20-d4ee9d38b826 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:56.911691 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99ae97b-4ecd-42f9-bf20-d4ee9d38b826 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:56.922693 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1446/2877] 10.222.0.61 () {58 vars in 1122 bytes} [Fri Aug 26 12:25:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 26 12:25:56.926691 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5558e3b-a19d-4b5a-9a0d-bd6cef8065db admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:56.937197 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5558e3b-a19d-4b5a-9a0d-bd6cef8065db admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:56.942043 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589c2dbb-f243-4273-842e-0f4e5acee250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:56.942688 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589c2dbb-f243-4273-842e-0f4e5acee250 admin admin] REQUEST_METHOD: `PUT` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:56.942969 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589c2dbb-f243-4273-842e-0f4e5acee250 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:56.943278 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589c2dbb-f243-4273-842e-0f4e5acee250 admin admin] PATH_INFO: `/v3/projects/9a6f1bf35060433bb91cdedf633ebcd8/users/b07c76d1e62a4f4290b276bc233d953c/roles/b2403690f5ad443aad059f5ee024f61c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:56.961838 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589c2dbb-f243-4273-842e-0f4e5acee250 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a6f1bf35060433bb91cdedf633ebcd8, user_id=b07c76d1e62a4f4290b276bc233d953c, role_id=b2403690f5ad443aad059f5ee024f61c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:56.967083 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589c2dbb-f243-4273-842e-0f4e5acee250 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:56.981266 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1432/2878] 10.222.0.61 () {60 vars in 1483 bytes} [Fri Aug 26 12:25:56 2022] PUT /identity/v3/projects/9a6f1bf35060433bb91cdedf633ebcd8/users/b07c76d1e62a4f4290b276bc233d953c/roles/b2403690f5ad443aad059f5ee024f61c => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:25:56.987524 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2dc7e4-24c6-48fb-8651-1bf11f98de91 None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:56.987814 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2dc7e4-24c6-48fb-8651-1bf11f98de91 None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:56.988092 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2dc7e4-24c6-48fb-8651-1bf11f98de91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:57.006852 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-8a2dc7e4-24c6-48fb-8651-1bf11f98de91 None None] MFA Rules not processed for user `d101ea12e0cd40e0b25022bf649ac734`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:25:57.008139 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-8a2dc7e4-24c6-48fb-8651-1bf11f98de91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:57.065448 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1447/2879] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:25:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 26 12:25:57.075162 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589c2dbb-f243-4273-842e-0f4e5acee250 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:57.076879 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-589c2dbb-f243-4273-842e-0f4e5acee250 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:57.099395 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:25:57.135257 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589c2dbb-f243-4273-842e-0f4e5acee250 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:57.138804 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c9b7f1-e0ad-4835-9d5a-f7a74c2642a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:57.139430 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9b7f1-e0ad-4835-9d5a-f7a74c2642a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:57.139706 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9b7f1-e0ad-4835-9d5a-f7a74c2642a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:57.139968 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c9b7f1-e0ad-4835-9d5a-f7a74c2642a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:57.149238 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c9b7f1-e0ad-4835-9d5a-f7a74c2642a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:57.150651 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c9b7f1-e0ad-4835-9d5a-f7a74c2642a3 service neutron] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:57.160416 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1433/2880] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:25:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:25:57.780699 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe92ed-59c2-4001-9eb3-ae42af7dceca None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:57.780991 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe92ed-59c2-4001-9eb3-ae42af7dceca None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:57.781300 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe92ed-59c2-4001-9eb3-ae42af7dceca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:57.810372 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-5bfe92ed-59c2-4001-9eb3-ae42af7dceca None None] MFA Rules not processed for user `b07c76d1e62a4f4290b276bc233d953c`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:25:57.811615 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-5bfe92ed-59c2-4001-9eb3-ae42af7dceca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:57.829090 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:25:57.880330 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1448/2881] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:25:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:25:57.891029 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c9b7f1-e0ad-4835-9d5a-f7a74c2642a3 service neutron] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:57.892446 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-30c9b7f1-e0ad-4835-9d5a-f7a74c2642a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:57.950981 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c9b7f1-e0ad-4835-9d5a-f7a74c2642a3 service neutron] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:57.954613 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a7c65-966d-4c6b-aeb6-f2b6fb1b911c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51239ac314e245e8a5706e65b133db3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:57.955269 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a7c65-966d-4c6b-aeb6-f2b6fb1b911c service cinder] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:57.955549 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a7c65-966d-4c6b-aeb6-f2b6fb1b911c service cinder] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:57.955803 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a7c65-966d-4c6b-aeb6-f2b6fb1b911c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:57.964188 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141a7c65-966d-4c6b-aeb6-f2b6fb1b911c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:57.965604 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141a7c65-966d-4c6b-aeb6-f2b6fb1b911c service cinder] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:57.974275 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1434/2882] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:25:57 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:25:59.319465 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868cf5cf-e962-422f-897f-2d11820f98dc None None] REQUEST_METHOD: `POST` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:59.320008 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868cf5cf-e962-422f-897f-2d11820f98dc None None] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:59.320008 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868cf5cf-e962-422f-897f-2d11820f98dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:59.339010 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.auth.core [None req-868cf5cf-e962-422f-897f-2d11820f98dc None None] MFA Rules not processed for user `b07c76d1e62a4f4290b276bc233d953c`. Rule list: `[]` (Enabled: `True`). {{(pid=81712) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:25:59.340226 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-868cf5cf-e962-422f-897f-2d11820f98dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:59.347688 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1449/2883] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:25:59 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:25:59.584100 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a7c65-966d-4c6b-aeb6-f2b6fb1b911c service cinder] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:25:59.586376 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-141a7c65-966d-4c6b-aeb6-f2b6fb1b911c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:59.599032 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a7c65-966d-4c6b-aeb6-f2b6fb1b911c service cinder] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:25:59.602530 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d19cb2-164f-4595-b735-7fe60c3f4d4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0972dbd5cff4610b239326425e6351c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:25:59.603094 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d19cb2-164f-4595-b735-7fe60c3f4d4b service neutron] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:25:59.603414 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d19cb2-164f-4595-b735-7fe60c3f4d4b service neutron] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:25:59.603686 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d19cb2-164f-4595-b735-7fe60c3f4d4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:25:59.605336 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-f6d19cb2-164f-4595-b735-7fe60c3f4d4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:25:59.614677 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d19cb2-164f-4595-b735-7fe60c3f4d4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:25:59.616100 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d19cb2-164f-4595-b735-7fe60c3f4d4b service neutron] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:25:59.623979 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1435/2884] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:25:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 26 12:26:07.076774 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868cf5cf-e962-422f-897f-2d11820f98dc None None] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:26:07.078294 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-868cf5cf-e962-422f-897f-2d11820f98dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:26:07.158009 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868cf5cf-e962-422f-897f-2d11820f98dc None None] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:26:07.164099 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3af261-c7b3-4e63-92cb-c44474c1f6e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aacd7d6d17e4f10ba5c9107a12beafa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:26:07.164705 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3af261-c7b3-4e63-92cb-c44474c1f6e9 service nova] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:26:07.164986 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3af261-c7b3-4e63-92cb-c44474c1f6e9 service nova] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:26:07.165258 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3af261-c7b3-4e63-92cb-c44474c1f6e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:26:07.166949 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-ce3af261-c7b3-4e63-92cb-c44474c1f6e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:26:07.179557 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3af261-c7b3-4e63-92cb-c44474c1f6e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:26:07.181191 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3af261-c7b3-4e63-92cb-c44474c1f6e9 service nova] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:26:07.192758 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1450/2885] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:26:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:26:09.620513 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=49 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:26:13.662734 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48caf62e-dd41-4991-8149-98438813872a None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:26:13.663362 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48caf62e-dd41-4991-8149-98438813872a None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:26:13.663362 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48caf62e-dd41-4991-8149-98438813872a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:26:13.681726 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-48caf62e-dd41-4991-8149-98438813872a None None] MFA Rules not processed for user `b07c76d1e62a4f4290b276bc233d953c`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:26:13.682908 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-48caf62e-dd41-4991-8149-98438813872a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:26:13.689813 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1436/2886] 10.222.0.61 () {58 vars in 958 bytes} [Fri Aug 26 12:26:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:26:16.686193 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3af261-c7b3-4e63-92cb-c44474c1f6e9 service nova] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:26:16.688984 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-ce3af261-c7b3-4e63-92cb-c44474c1f6e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:26:16.706488 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3af261-c7b3-4e63-92cb-c44474c1f6e9 service nova] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:26:16.713462 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1e0d16-2295-4edb-a374-2e863d1c857b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7aacd7d6d17e4f10ba5c9107a12beafa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:26:16.714095 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e0d16-2295-4edb-a374-2e863d1c857b service nova] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:26:16.714373 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e0d16-2295-4edb-a374-2e863d1c857b service nova] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:26:16.714850 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e0d16-2295-4edb-a374-2e863d1c857b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:26:16.716874 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-5e1e0d16-2295-4edb-a374-2e863d1c857b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:26:16.731358 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1e0d16-2295-4edb-a374-2e863d1c857b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:26:16.733145 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1e0d16-2295-4edb-a374-2e863d1c857b service nova] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:26:16.756907 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1451/2887] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:26:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:26:23.676723 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=3 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:26:25.034196 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48caf62e-dd41-4991-8149-98438813872a None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:26:25.036037 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-48caf62e-dd41-4991-8149-98438813872a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:26:25.159314 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48caf62e-dd41-4991-8149-98438813872a None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:26:25.164381 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab543e66-5089-46a7-a448-aceb568360fb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7346e4a29dfb4a0a9768597704be286c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:26:25.164997 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab543e66-5089-46a7-a448-aceb568360fb service placement] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:26:25.165281 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab543e66-5089-46a7-a448-aceb568360fb service placement] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:26:25.165558 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab543e66-5089-46a7-a448-aceb568360fb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:26:25.167305 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-ab543e66-5089-46a7-a448-aceb568360fb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:26:25.180157 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab543e66-5089-46a7-a448-aceb568360fb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:26:25.181792 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab543e66-5089-46a7-a448-aceb568360fb service placement] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:26:25.193489 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1437/2888] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:26:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:26:26.751041 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=62 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:26:28.503373 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1e0d16-2295-4edb-a374-2e863d1c857b service nova] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:26:28.518259 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1e0d16-2295-4edb-a374-2e863d1c857b service nova] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:26:28.523863 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99bd265-4c82-4099-8c50-5ca02286da1f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7346e4a29dfb4a0a9768597704be286c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:26:28.525785 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99bd265-4c82-4099-8c50-5ca02286da1f service placement] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:26:28.526479 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99bd265-4c82-4099-8c50-5ca02286da1f service placement] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:26:28.527157 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99bd265-4c82-4099-8c50-5ca02286da1f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:26:28.529850 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-b99bd265-4c82-4099-8c50-5ca02286da1f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:26:28.548277 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99bd265-4c82-4099-8c50-5ca02286da1f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:26:28.554646 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99bd265-4c82-4099-8c50-5ca02286da1f service placement] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:26:28.583865 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1452/2889] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:26:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:26:28.594495 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab543e66-5089-46a7-a448-aceb568360fb service placement] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:26:28.596717 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-ab543e66-5089-46a7-a448-aceb568360fb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:26:28.615261 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab543e66-5089-46a7-a448-aceb568360fb service placement] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:26:28.623609 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb32525e-5ef8-4669-99f2-4c7a82f1eb41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:26:28.624513 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32525e-5ef8-4669-99f2-4c7a82f1eb41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:26:28.625337 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32525e-5ef8-4669-99f2-4c7a82f1eb41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:26:28.625680 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32525e-5ef8-4669-99f2-4c7a82f1eb41 admin admin] PATH_INFO: `/v3/users/4e949eeed6cd451d8dc5d0877963b260` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:26:28.627076 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb32525e-5ef8-4669-99f2-4c7a82f1eb41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e949eeed6cd451d8dc5d0877963b260)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:26:28.629199 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb32525e-5ef8-4669-99f2-4c7a82f1eb41 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:26:28.684762 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-bb32525e-5ef8-4669-99f2-4c7a82f1eb41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e949eeed6cd451d8dc5d0877963b260'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:26:28.693421 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-bb32525e-5ef8-4669-99f2-4c7a82f1eb41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e949eeed6cd451d8dc5d0877963b260'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:26:28.700792 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-bb32525e-5ef8-4669-99f2-4c7a82f1eb41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e949eeed6cd451d8dc5d0877963b260'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:26:28.701435 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-bb32525e-5ef8-4669-99f2-4c7a82f1eb41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e949eeed6cd451d8dc5d0877963b260'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:26:28.718736 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-bb32525e-5ef8-4669-99f2-4c7a82f1eb41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e949eeed6cd451d8dc5d0877963b260'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:26:28.725520 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1438/2890] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:26:28 2022] DELETE /identity/v3/users/4e949eeed6cd451d8dc5d0877963b260 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:26:28.911243 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99bd265-4c82-4099-8c50-5ca02286da1f service placement] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:26:28.934246 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99bd265-4c82-4099-8c50-5ca02286da1f service placement] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:26:28.997128 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fc62b9-ce9f-427c-b0fc-cff04ad084c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:26:28.997791 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc62b9-ce9f-427c-b0fc-cff04ad084c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:26:28.998068 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc62b9-ce9f-427c-b0fc-cff04ad084c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:26:28.998338 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc62b9-ce9f-427c-b0fc-cff04ad084c8 admin admin] PATH_INFO: `/v3/projects/f885c9fd8f394c999eee70e8d27180e0` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:26:28.999519 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fc62b9-ce9f-427c-b0fc-cff04ad084c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f885c9fd8f394c999eee70e8d27180e0)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:26:29.002081 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fc62b9-ce9f-427c-b0fc-cff04ad084c8 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:26:29.051091 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-b8fc62b9-ce9f-427c-b0fc-cff04ad084c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f885c9fd8f394c999eee70e8d27180e0', 'request_id': 'req-b8fc62b9-ce9f-427c-b0fc-cff04ad084c8'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:26:29.057495 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-b8fc62b9-ce9f-427c-b0fc-cff04ad084c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f885c9fd8f394c999eee70e8d27180e0', 'request_id': 'req-b8fc62b9-ce9f-427c-b0fc-cff04ad084c8'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:26:29.065530 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-b8fc62b9-ce9f-427c-b0fc-cff04ad084c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f885c9fd8f394c999eee70e8d27180e0', 'request_id': 'req-b8fc62b9-ce9f-427c-b0fc-cff04ad084c8'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:26:29.076075 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-b8fc62b9-ce9f-427c-b0fc-cff04ad084c8 admin admin] The token cache is being invalidate because project f885c9fd8f394c999eee70e8d27180e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81712) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:26:29.076362 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-b8fc62b9-ce9f-427c-b0fc-cff04ad084c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:26:29.077170 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-b8fc62b9-ce9f-427c-b0fc-cff04ad084c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:26:29.079543 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1453/2891] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:26:28 2022] DELETE /identity/v3/projects/f885c9fd8f394c999eee70e8d27180e0 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:26:29.083287 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb32525e-5ef8-4669-99f2-4c7a82f1eb41 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:26:29.084725 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-bb32525e-5ef8-4669-99f2-4c7a82f1eb41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:26:29.097845 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb32525e-5ef8-4669-99f2-4c7a82f1eb41 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:26:29.156158 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e03aa7-62be-41e8-9148-d91be78d777c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:26:29.157393 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e03aa7-62be-41e8-9148-d91be78d777c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:26:29.157767 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e03aa7-62be-41e8-9148-d91be78d777c admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:26:29.158040 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e03aa7-62be-41e8-9148-d91be78d777c admin admin] PATH_INFO: `/v3/users/f9bc3cca053e402e819e386d57b66a7c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:26:29.159196 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e03aa7-62be-41e8-9148-d91be78d777c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9bc3cca053e402e819e386d57b66a7c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:26:29.164598 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e03aa7-62be-41e8-9148-d91be78d777c admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:26:29.164794 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:26:29.223363 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-16e03aa7-62be-41e8-9148-d91be78d777c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9bc3cca053e402e819e386d57b66a7c'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:26:29.232497 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-16e03aa7-62be-41e8-9148-d91be78d777c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9bc3cca053e402e819e386d57b66a7c'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:26:29.240861 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-16e03aa7-62be-41e8-9148-d91be78d777c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9bc3cca053e402e819e386d57b66a7c'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:26:29.241578 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-16e03aa7-62be-41e8-9148-d91be78d777c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9bc3cca053e402e819e386d57b66a7c'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:26:29.256247 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-16e03aa7-62be-41e8-9148-d91be78d777c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9bc3cca053e402e819e386d57b66a7c'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:26:29.271116 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1439/2892] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:26:29 2022] DELETE /identity/v3/users/f9bc3cca053e402e819e386d57b66a7c => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:26:29.453746 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fc62b9-ce9f-427c-b0fc-cff04ad084c8 admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:26:29.456094 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-b8fc62b9-ce9f-427c-b0fc-cff04ad084c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:26:29.487193 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fc62b9-ce9f-427c-b0fc-cff04ad084c8 admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:26:29.546557 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f488eba-4ac7-4475-a43d-4b590e7fac9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:26:29.547838 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f488eba-4ac7-4475-a43d-4b590e7fac9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:26:29.548011 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f488eba-4ac7-4475-a43d-4b590e7fac9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:26:29.548375 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f488eba-4ac7-4475-a43d-4b590e7fac9f admin admin] PATH_INFO: `/v3/projects/c7fa9fc32592449ebfda1144f8887475` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:26:29.549493 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f488eba-4ac7-4475-a43d-4b590e7fac9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7fa9fc32592449ebfda1144f8887475)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:26:29.551432 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f488eba-4ac7-4475-a43d-4b590e7fac9f admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:26:29.573597 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:26:29.601506 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-1f488eba-4ac7-4475-a43d-4b590e7fac9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7fa9fc32592449ebfda1144f8887475', 'request_id': 'req-1f488eba-4ac7-4475-a43d-4b590e7fac9f'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:26:29.607642 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-1f488eba-4ac7-4475-a43d-4b590e7fac9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7fa9fc32592449ebfda1144f8887475', 'request_id': 'req-1f488eba-4ac7-4475-a43d-4b590e7fac9f'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:26:29.615930 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-1f488eba-4ac7-4475-a43d-4b590e7fac9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7fa9fc32592449ebfda1144f8887475', 'request_id': 'req-1f488eba-4ac7-4475-a43d-4b590e7fac9f'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:26:29.626263 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-1f488eba-4ac7-4475-a43d-4b590e7fac9f admin admin] The token cache is being invalidate because project c7fa9fc32592449ebfda1144f8887475 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81712) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:26:29.626546 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-1f488eba-4ac7-4475-a43d-4b590e7fac9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:26:29.627166 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-1f488eba-4ac7-4475-a43d-4b590e7fac9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:26:29.628734 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1454/2893] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:26:29 2022] DELETE /identity/v3/projects/c7fa9fc32592449ebfda1144f8887475 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:26:35.458207 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0677e09e-fab5-4618-8b43-a033f85e9b57 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:26:35.458663 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0677e09e-fab5-4618-8b43-a033f85e9b57 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:26:35.458777 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0677e09e-fab5-4618-8b43-a033f85e9b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:26:35.507454 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-0677e09e-fab5-4618-8b43-a033f85e9b57 None None] MFA Rules not processed for user `755b04cae45746569bfbf92f2f213a58`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:26:35.508658 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-0677e09e-fab5-4618-8b43-a033f85e9b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:26:35.573630 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1440/2894] 10.222.0.61 () {58 vars in 1009 bytes} [Fri Aug 26 12:26:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:26:35.620794 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f488eba-4ac7-4475-a43d-4b590e7fac9f admin admin] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:26:35.622257 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.fernet_utils [None req-1f488eba-4ac7-4475-a43d-4b590e7fac9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81712) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:26:35.711793 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f488eba-4ac7-4475-a43d-4b590e7fac9f admin admin] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:26:35.717181 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f179fad-4994-429c-a517-b078f3691088 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead95a244d11432d9fe9f718e1a75f38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:26:35.717791 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f179fad-4994-429c-a517-b078f3691088 service swift] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:26:35.718066 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f179fad-4994-429c-a517-b078f3691088 service swift] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:26:35.718336 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f179fad-4994-429c-a517-b078f3691088 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:26:35.727120 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f179fad-4994-429c-a517-b078f3691088 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:26:35.728713 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f179fad-4994-429c-a517-b078f3691088 service swift] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:26:35.737526 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1455/2895] 10.222.0.61 () {60 vars in 1384 bytes} [Fri Aug 26 12:26:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 26 12:26:45.570802 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=65,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:26:45.733536 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:26:50.177640 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df60a4-c829-4ea8-9c76-626e12d97882 None None] REQUEST_METHOD: `POST` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:26:50.178108 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df60a4-c829-4ea8-9c76-626e12d97882 None None] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:26:50.178214 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df60a4-c829-4ea8-9c76-626e12d97882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:26:50.196562 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.auth.core [None req-75df60a4-c829-4ea8-9c76-626e12d97882 None None] MFA Rules not processed for user `755b04cae45746569bfbf92f2f213a58`. Rule list: `[]` (Enabled: `True`). {{(pid=81713) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 26 12:26:50.197747 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-75df60a4-c829-4ea8-9c76-626e12d97882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:26:50.205308 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1441/2896] 10.222.0.61 () {58 vars in 967 bytes} [Fri Aug 26 12:26:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 26 12:26:50.217165 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f179fad-4994-429c-a517-b078f3691088 service swift] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:26:50.228682 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f179fad-4994-429c-a517-b078f3691088 service swift] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:26:50.233718 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7f3c6a-287e-4cc1-afae-10b4adb43c66 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ead95a244d11432d9fe9f718e1a75f38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:26:50.234573 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f3c6a-287e-4cc1-afae-10b4adb43c66 service swift] REQUEST_METHOD: `GET` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:26:50.234892 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f3c6a-287e-4cc1-afae-10b4adb43c66 service swift] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:26:50.235349 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f3c6a-287e-4cc1-afae-10b4adb43c66 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:26:50.243225 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7f3c6a-287e-4cc1-afae-10b4adb43c66 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:26:50.244789 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7f3c6a-287e-4cc1-afae-10b4adb43c66 service swift] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:26:50.252692 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1456/2897] 10.222.0.61 () {60 vars in 1384 bytes} [Fri Aug 26 12:26:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 26 12:26:52.152412 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75df60a4-c829-4ea8-9c76-626e12d97882 None None] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:26:52.153951 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-75df60a4-c829-4ea8-9c76-626e12d97882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:26:52.221983 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75df60a4-c829-4ea8-9c76-626e12d97882 None None] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:26:52.226346 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3fa48b-9fe7-47d5-bc33-1b6b5b913861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:26:52.226959 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3fa48b-9fe7-47d5-bc33-1b6b5b913861 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:26:52.227261 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3fa48b-9fe7-47d5-bc33-1b6b5b913861 admin admin] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:26:52.227534 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3fa48b-9fe7-47d5-bc33-1b6b5b913861 admin admin] PATH_INFO: `/v3/users/b07c76d1e62a4f4290b276bc233d953c` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:26:52.228638 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3fa48b-9fe7-47d5-bc33-1b6b5b913861 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b07c76d1e62a4f4290b276bc233d953c)` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:26:52.230532 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3fa48b-9fe7-47d5-bc33-1b6b5b913861 admin admin] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:26:52.274963 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-7e3fa48b-9fe7-47d5-bc33-1b6b5b913861 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b07c76d1e62a4f4290b276bc233d953c'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:26:52.281509 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-7e3fa48b-9fe7-47d5-bc33-1b6b5b913861 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b07c76d1e62a4f4290b276bc233d953c'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:26:52.288560 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-7e3fa48b-9fe7-47d5-bc33-1b6b5b913861 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b07c76d1e62a4f4290b276bc233d953c'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:26:52.289187 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-7e3fa48b-9fe7-47d5-bc33-1b6b5b913861 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b07c76d1e62a4f4290b276bc233d953c'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:26:52.300731 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.notifications [None req-7e3fa48b-9fe7-47d5-bc33-1b6b5b913861 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b07c76d1e62a4f4290b276bc233d953c'} {{(pid=81713) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:26:52.306751 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1442/2898] 10.222.0.61 () {58 vars in 1224 bytes} [Fri Aug 26 12:26:52 2022] DELETE /identity/v3/users/b07c76d1e62a4f4290b276bc233d953c => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:26:52.475794 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7f3c6a-287e-4cc1-afae-10b4adb43c66 service swift] Authenticating user token {{(pid=81712) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:26:52.488213 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7f3c6a-287e-4cc1-afae-10b4adb43c66 service swift] Validating token access rules against request {{(pid=81712) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:26:52.543534 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6180b99-f84c-4e24-95ff-9fa79c329dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd101ea12e0cd40e0b25022bf649ac734', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b98963fff98a48a1b4cae11131b453da', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81712) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:26:52.544154 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6180b99-f84c-4e24-95ff-9fa79c329dd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:26:52.544427 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6180b99-f84c-4e24-95ff-9fa79c329dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:26:52.544695 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6180b99-f84c-4e24-95ff-9fa79c329dd8 admin admin] PATH_INFO: `/v3/projects/9a6f1bf35060433bb91cdedf633ebcd8` {{(pid=81712) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:26:52.545786 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6180b99-f84c-4e24-95ff-9fa79c329dd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a6f1bf35060433bb91cdedf633ebcd8)` {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:26:52.547699 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6180b99-f84c-4e24-95ff-9fa79c329dd8 admin admin] RBAC: Authorization granted {{(pid=81712) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:26:52.592963 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-a6180b99-f84c-4e24-95ff-9fa79c329dd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a6f1bf35060433bb91cdedf633ebcd8', 'request_id': 'req-a6180b99-f84c-4e24-95ff-9fa79c329dd8'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:26:52.598952 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-a6180b99-f84c-4e24-95ff-9fa79c329dd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a6f1bf35060433bb91cdedf633ebcd8', 'request_id': 'req-a6180b99-f84c-4e24-95ff-9fa79c329dd8'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:26:52.606301 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-a6180b99-f84c-4e24-95ff-9fa79c329dd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a6f1bf35060433bb91cdedf633ebcd8', 'request_id': 'req-a6180b99-f84c-4e24-95ff-9fa79c329dd8'} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:26:52.617850 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-a6180b99-f84c-4e24-95ff-9fa79c329dd8 admin admin] The token cache is being invalidate because project 9a6f1bf35060433bb91cdedf633ebcd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81712) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 26 12:26:52.618136 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-a6180b99-f84c-4e24-95ff-9fa79c329dd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:26:52.618753 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG keystone.notifications [None req-a6180b99-f84c-4e24-95ff-9fa79c329dd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81712) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 26 12:26:52.620244 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: [pid: 81712|app: 0|req: 1457/2899] 10.222.0.61 () {58 vars in 1233 bytes} [Fri Aug 26 12:26:52 2022] DELETE /identity/v3/projects/9a6f1bf35060433bb91cdedf633ebcd8 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 26 12:27:02.302971 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=56,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:27:02.614898 dv-cinder-846303-2-iscsi devstack@keystone.service[81712]: DEBUG dbcounter [-] [81712] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81712) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 26 12:28:20.302844 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3fa48b-9fe7-47d5-bc33-1b6b5b913861 admin admin] Authenticating user token {{(pid=81713) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 26 12:28:20.304390 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-7e3fa48b-9fe7-47d5-bc33-1b6b5b913861 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:28:20.372483 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3fa48b-9fe7-47d5-bc33-1b6b5b913861 admin admin] Validating token access rules against request {{(pid=81713) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 26 12:28:20.431712 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f96046d-89bc-4661-9a84-babf52144f67 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7346e4a29dfb4a0a9768597704be286c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ecd68423a83a427681d44dbf266fce6e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81713) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 26 12:28:20.432317 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f96046d-89bc-4661-9a84-babf52144f67 service placement] REQUEST_METHOD: `GET` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 26 12:28:20.432599 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f96046d-89bc-4661-9a84-babf52144f67 service placement] SCRIPT_NAME: `/identity` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 26 12:28:20.432904 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f96046d-89bc-4661-9a84-babf52144f67 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81713) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 26 12:28:20.434878 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.fernet_utils [None req-2f96046d-89bc-4661-9a84-babf52144f67 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81713) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 26 12:28:20.446692 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f96046d-89bc-4661-9a84-babf52144f67 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 26 12:28:20.448310 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f96046d-89bc-4661-9a84-babf52144f67 service placement] RBAC: Authorization granted {{(pid=81713) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 26 12:28:20.458647 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: [pid: 81713|app: 0|req: 1443/2900] 10.222.0.61 () {60 vars in 1365 bytes} [Fri Aug 26 12:28:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 26 12:28:30.454167 dv-cinder-846303-2-iscsi devstack@keystone.service[81713]: DEBUG dbcounter [-] [81713] Writing DB stats keystone:SELECT=58 {{(pid=81713) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}